Using Incremental PLSI for Threshold-Resilient Online Event Analysis
Authors:
- 31 January 2008
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 20 (3) , 289-299
Abstract
The goal of online event analysis is to detect events and track their associated documents in real time from a continuous stream of documents generated by multiple information sources. Unlike traditional text categorization methods, event analysis approaches consider the temporal relations among documents. However, such methods suffer from the threshold-dependency problem, so they only perform well for a narrow range of thresholds. In addition, if the contents of a document stream change, the optimal threshold (that is, the threshold that yields the best performance) often changes as well. In this paper, we propose a threshold-resilient online algorithm, called the incremental probabilistic latent semantic indexing (IPLSI) algorithm, which alleviates the threshold-dependency problem and simultaneously maintains the continuity of the latent semantics to better capture the story line development of events. The IPLSI algorithm is theoretically sound and empirically efficient and effective for event analysis. The results of the performance evaluation performed on the topic detection and tracking (TDT)-4 corpus show that the algorithm reduces the cost of event analysis by as much as 15 percent ~ 20 percent and increases the acceptable threshold range by 200 percent to 300 percent over the baseline.Keywords
This publication has 16 references indexed in Scilit:
- An Aging Theory for Event Life-Cycle ModelingIEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 2007
- Evolutionary clustering2006
- A probabilistic model for retrospective news event detection2005
- Web usage mining based on probabilistic latent semantic analysis2004
- A System for new event detection2003
- A critical examination of TDT's cost function2002
- Topic-based document segmentation with probabilistic latent semantic analysis2002
- Topic segmentation with an aspect hidden Markov model2001
- Probabilistic latent semantic indexing1999
- On-line new event detection and tracking1998