A Coincidence-Based Test for Uniformity Given Very Sparsely Sampled Discrete Data
Author:
- 16 September 2008
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 54 (10) , 4750-4755
Abstract
How many independent samples N do we need from a distribution p to decide that p is epsiv-distant from uniform in an L1 sense, Sigmai=1 m |p(i) - 1/m| > epsiv? (Here m is the number of bins on which the distribution is supported, and is assumed known a priori.) Somewhat surprisingly, we only need N epsiv2 Gt m 1/2 to make this decision reliably (this condition is both sufficient and necessary). The test for uniformity introduced here is based on the number of observed ldquocoincidencesrdquo (samples that fall into the same bin), the mean and variance of which may be computed explicitly for the uniform distribution and bounded nonparametrically for any distribution that is known to be epsiv-distant from uniform. Some connections to the classical birthday problem are noted.Keywords
This publication has 11 references indexed in Scilit:
- The matching, birthday and the strong birthday problem: a contemporary reviewJournal of Statistical Planning and Inference, 2005
- Estimating Entropy on$m$Bins Given Fewer Than$m$SamplesIEEE Transactions on Information Theory, 2004
- Entropy and information in neural spike trains: Progress on the sampling problemPhysical Review E, 2004
- Limit Distributions and Random Trees Derived from the Birthday Problem with Unequal ProbabilitiesElectronic Journal of Probability, 2000
- Majorization and the Birthday InequalityMathematics Magazine, 1991
- Achieving Information Bounds in Non and Semiparametric ModelsThe Annals of Statistics, 1990
- Methods for Studying CoincidencesJournal of the American Statistical Association, 1989
- An Efron-Stein Inequality for Nonsymmetric StatisticsThe Annals of Statistics, 1986
- A Note on the Uniformity Assumption in the Birthday ProblemThe American Statistician, 1977
- E2386The American Mathematical Monthly, 1973