Semi-parametric Robust Event Detection for Massive Time-Domain Databases
Preprint
Abstract
The detection and analysis of events within massive collections of time-series has become an extremely important task for time-domain astronomy. In particular, many scientific investigations (e.g. the analysis of microlensing and other transients) begin with the detection of isolated events in irregularly-sampled series with both non-linear trends and non-Gaussian noise. We outline a semi-parametric, robust, parallel method for identifying variability and isolated events at multiple scales in the presence of the above complications. This approach harnesses the power of Bayesian modeling while maintaining much of the speed and scalability of more ad-hoc machine learning approaches. We also contrast this work with event detection methods from other fields, highlighting the unique challenges posed by astronomical surveys. Finally, we present results from the application of this method to 87.2 million EROS-2 sources, where we have obtained a greater than 100-fold reduction in candidates for certain types of phenomena while creating high-quality features for subsequent analyses.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Proceedings of the 2009 SIAM International Conference on Data Mining2009
- A weakly informative default prior distribution for logistic and other regression modelsThe Annals of Applied Statistics, 2008
- Learning to detect events with Markov-modulated poisson processesACM Transactions on Knowledge Discovery From Data, 2007
- Estimating restricted structural change modelsJournal of Econometrics, 2006
- Statistical Methods for Detecting Stellar Occultations by Kuiper Belt Objects: The Taiwanese–American Occultation SurveyStatistical Science, 2004
- The MACHO Project: Microlensing Detection EfficiencyThe Astrophysical Journal Supplement Series, 2001
- The control of the false discovery rate in multiple testing under dependencyThe Annals of Statistics, 2001
- Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple TestingJournal of the Royal Statistical Society Series B: Statistical Methodology, 1995
- Hierarchical Bayesian Analysis of Changepoint ProblemsJournal of the Royal Statistical Society Series C: Applied Statistics, 1992
- Maximum Likelihood from Incomplete Data Via the EM AlgorithmJournal of the Royal Statistical Society Series B: Statistical Methodology, 1977