Optimal and fast detection of spatial clusters with scan statistics
Author:
- Guenther WaltherGWGuenther Walther
Preprint
Abstract
We consider the detection of multivariate spatial clusters in the Bernoulli model with $N$ locations, where the design distribution has weakly dependent marginals. The locations are scanned with a rectangular window with sides parallel to the axes and with varying sizes and aspect ratios. Multivariate scan statistics pose a statistical problem due to the multiple testing over many scan windows, as well as a computational problem because statistics have to be evaluated on many windows. This paper introduces methodology that leads to both statistically optimal inference and computationally efficient algorithms. The main difference to the traditional calibration of scan statistics is the concept of grouping scan windows according to their sizes, and then applying different critical values to different groups. It is shown that this calibration of the scan statistic results in optimal inference for spatial clusters on both small scales and on large scales, as well as in the case where the cluster lives on one of the marginals. Methodology is introduced that allows for an efficient approximation of the set of all rectangles while still guaranteeing the statistical optimality results described above. It is shown that the resulting scan statistic has a computational complexity that is almost linear in $N$.
Keywords
All Related Versions
- Version 1, 2010-02-25, ArXiv
- Published version: The Annals of Statistics, 38 (2), 1010.
This publication has 11 references indexed in Scilit:
- The Block Criterion for Multiscale Inference About a Density, With Applications to Other Multiscale ProblemsJournal of Computational and Graphical Statistics, 2010
- Multiscale inference about a densityThe Annals of Statistics, 2008
- Explicit Learning Curves for Transduction and Application to Clustering and Compression AlgorithmsJournal of Artificial Intelligence Research, 2004
- Frequency difference gating: A multivariate method for identifying subsets that differ between samplesCytometry, 2001
- Computing Scan StatisticpValues Using Importance Sampling, With Applications to Genetics and Medical Image AnalysisJournal of Computational and Graphical Statistics, 2001
- Multiscale Testing of Qualitative HypothesesThe Annals of Statistics, 2001
- Asymptotically exact nonparametric hypothesis testing in sup-norm and at a fixed pointProbability Theory and Related Fields, 2000
- Some Methods for Investigating Spatial Clustering, with Epidemiological ApplicationsJournal of the Royal Statistical Society Series A: Statistics in Society, 1997
- Two-dimensional discrete scan statisticsStatistics & Probability Letters, 1996
- Clustering of Random Points in Two DimensionsBiometrika, 1965