Opening book details…
Can I read Improved Language Modelling Through Better Language Model Evaluation Measures on EtoBox?
Improved Language Modelling Through Better Language Model Evaluation Measures by Philip Clarkson; Tony Robinson is a Computer Science article available to read on EtoBox.
What is Improved Language Modelling Through Better Language Model Evaluation Measures about?
This paper explores the interaction between a language model's perplexity and its effect on the word error rate of a speech recognition system. Much recent research has indicated that these two measures are not as well correlated as was once thought, and many examples exist of models which have a much lower perplexity than the equivalent N -gram model, yet lead to no improvement in recognition accuracy. This paper investigates the reasons for this apparent discrepancy. Perplexity's calculation is based solely on the probabilities of words contained within the test text; it disregards the probabilities of alternative words which will be competing with the correct word within the decoder. It is shown that by considering the probabilities of the alternative words it is possible to derive measures of language model quality which are better correlated with word error rate than perplexity is. Furthermore, optimizing language model parameters with respect to these new measures leads to a significant reduction in the word error rate.
Who reads Improved Language Modelling Through Better Language Model Evaluation Measures?
It is typically read by researchers, students, and practitioners in Computer Science.
- Author
- Philip Clarkson; Tony Robinson
- Publisher
- Elsevier Science; Elsevier ; Elsevier Inc.; Elsevier BV (ISSN 0885-2308)
- Published
- 2001
- Language
- EN
- Field
- Computer Science (Physical Sciences)