Statistical language modelling of dialogue material in the British national corpus
Statistical language modelling may not only be used to uncover the patterns which underlie the composition of utterances and texts, but also to build practical language processing technology. Contemporary language applications in automatic speech recognition, sentence interpretation and even machine translation exploit statistical models of language. Spoken dialogue systems, where a human user interacts with a machine via a speech interface in order to get information, make bookings, complaints, etc., are example of such systems which are now technologically feasible. The majority of statistical language modelling studies to date have concentrated on written text material (or read versions thereof). However, it is well-known that dialogue is significantly different from written text in its lexical content and sentence structure. Furthermore, there are expected to be significant logical, thematic and lexical connections between successive turns within a dialogue, but ~turns~ are not generally meaningful in written text. There is therefore a need for statistical language modeling studies to be performed on dialogue, particularly with a longer-term aim to using such models in human-machine dialogue interfaces.
Hunter, Gordon James Allan
Statistical language modelling of dialogue material in the British national corpus. Statistical language modelling may not only be used to uncover the patterns which underlie the composition of utterances and texts, but also to build practical language processing technology. Contemporary language applications in automatic speech recognition, sentence interpretation and even machine translation exploit statistical models of language. Spoken dialogue systems, where a human user interacts with a machine via a speech interface in order to get information, make bookings, complaints, etc., are example of such systems which are now technologically feasible. The majority of statistical language modelling studies to date have concentrated on written text material (or read versions thereof). However, it is well-known that dialogue is significantly different from written text in its lexical content and sentence structure. Furthermore, there are expected to be significant logical, thematic and lexical connections between successive turns within a dialogue, but ~turns~ are not generally meaningful in written text. There is therefore a need for statistical language modeling studies to be performed on dialogue, particularly with a longer-term aim to using such models in human-machine dialogue interfaces.
statistical language model_dialogue material british national corpus statistical language modelling may_use uncover pattern underlie composition utterance text also build practical language_processing technology contemporary language application automatic_speech recognition sentence interpretation even machine_translation exploit statistical_model language spoken dialogue_system human user machine via speech interface order get information make complaint etc example system technologically feasible majority statistical language model study date concentrate write_text material read version thereof however well_known dialogue significantly_different write_text lexical content sentence_structure furthermore expect significant logical thematic lexical connection successive turn within dialogue turns generally meaningful write_text therefore need statistical language model study perform dialogue particularly longer_term aim use model human machine dialogue interface