Statistical Language Models for Information Retrieval
Author:
- ChengXiang ZhaiCZChengXiang Zhai
- 1 January 2008
- journal article
- Published by Springer Nature in Synthesis Lectures on Human Language Technologies
- Vol. 1 (1) , 1-141
Abstract
As online information grows dramatically, search engines such as Google are playing a more and more important role in our lives. Critical to all search engines is the problem of designing an effective retrieval model that can rank documents accurately for a given query. This has been a central research problem in information retrieval for several decades. In the past ten years, a new generation of retrieval models, often referred to as statistical language models, has been successfully applied to solve many different information retrieval problems. Compared with the traditional models such as the vector space model, these new models have a more sound statistical foundation and can leverage statistical estimation to optimize retrieval parameters. They can also be more easily adapted to model non-traditional and complex retrieval problems. Empirically, they tend to achieve comparable or better performance than a traditional model with less effort on parameter tuning. This book systematically review...Keywords
This publication has 72 references indexed in Scilit:
- Statistical Language Models for Information Retrieval A Critical ReviewFoundations and Trends® in Information Retrieval, 2008
- Learning to rank for information retrieval (LR4IR 2007)ACM SIGIR Forum, 2007
- Pattern Recognition and Machine LearningJournal of Electronic Imaging, 2007
- A probabilistic justification for using tf×idf term weighting in information retrievalInternational Journal on Digital Libraries, 2000
- Divergence measures based on the Shannon entropyIEEE Transactions on Information Theory, 1991
- Indexing by latent semantic analysisJournal of the American Society for Information Science, 1990
- A tutorial on hidden Markov models and selected applications in speech recognitionProceedings of the IEEE, 1989
- Foundations of Probabilistic and Utility-Theoretic IndexingJournal of the ACM, 1978
- THE PROBABILITY RANKING PRINCIPLE IN IRJournal of Documentation, 1977
- A STATISTICAL INTERPRETATION OF TERM SPECIFICITY AND ITS APPLICATION IN RETRIEVALJournal of Documentation, 1972