Finite-Sample Maximum Likelihood Estimation of Location
Preprint
Abstract
We consider 1-dimensional location estimation, where we estimate a parameter $\lambda$ from $n$ samples $\lambda + \eta_i$, with each $\eta_i$ drawn i.i.d. from a known distribution $f$. For fixed $f$ the maximum-likelihood estimate (MLE) is well-known to be optimal in the limit as $n \to \infty$: it is asymptotically normal with variance matching the Cram\'er-Rao lower bound of $\frac{1}{n\mathcal{I}}$, where $\mathcal{I}$ is the Fisher information of $f$. However, this bound does not hold for finite $n$, or when $f$ varies with $n$. We show for arbitrary $f$ and $n$ that one can recover a similar theory based on the Fisher information of a smoothed version of $f$, where the smoothing radius decays with $n$.
Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- U-statistics of growing order and sub-Gaussian mean estimators with sharp constantsMathematical Statistics and Learning, 2023
- Optimal-order uniform and nonuniform bounds on the rate of convergence to normality for maximum likelihood estimatorsElectronic Journal of Statistics, 2017
- Parametric estimation. Finite sample theoryThe Annals of Statistics, 2012
- Challenging the empirical mean and empirical variance: A deviation studyAnnales de l'Institut Henri Poincaré, Probabilités et Statistiques, 2012
- Concentration inequality of maximum likelihood estimatorApplied Mathematics Letters, 2010
- Estimation des densit s: risque minimaxProbability Theory and Related Fields, 1979