Abstract
The development of a speech understanding grammar for spoken dialogue systems can be greatly accelerated by using an in-domain corpus. The development of such a corpus, however, is a slow and expensive process. This paper proposes unsupervised, language-agnostic methods for finding relevant corpora in the web and mining the most informative parts. We show that by utilizing perplexity we are able to increase the in-domainess (precision) of the mined corpora, while by utilizing pragmatic constraints and search engine rank we can increase the generalizability (recall). We show that automatic grammar induction algorithms achieve superior performance on the automatically mined corpora compared to in-domain manually collected corpora for a travel application.
Original language | English |
---|---|
Pages (from-to) | 2733-2737 |
Number of pages | 5 |
Journal | Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH |
Publication status | Published - 2013 |
Externally published | Yes |
Event | 14th Annual Conference of the International Speech Communication Association, INTERSPEECH 2013 - Lyon, France Duration: 25 Aug 2013 → 29 Aug 2013 |
Keywords
- Grammar induction
- Language modeling
- Speech understanding
- Spoken dialog systems
- Web harvesting