The role of likelihood and entropy in incomplete-data problems: Applications to estimating point-process intensities and toeplitz constrained covariances
Authors:
- D.L. SnyderDSD.L. Snyder
- 1 July 1987
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Proceedings of the IEEE
- Vol. 75 (7) , 892-907
Abstract
The principle of maximum entropy has played an important role in the solution of problems in which the measurements correspond to moment constraints on some many-to-one mapping h(x). In this paper we explore its role in estimation problems in which the measured data are statistical observations and moment constraints on the observation function h(x) do not exist. We conclude that: 1) For the class of likelihood problems arising in a complete-incomplete data context in which the complete data x are nonuniquely determined by the measured incomplete data y via the many-to-one mapping y = h(x), the density maximizing entropy is identical to the conditional density of the complete data given the incomplete data. This equivalence results by viewing the measurements as specifying the domain over which the density is defined, rather than as a moment constraint on h(x). 2) The identity between the maximum entropy and the conditional density results in the fact that maximum-likelihood estimates may be obtained via a joint maximization (minimization) of the entropy function (Kullback-Liebler divergence). This provides the basis for the iterative algorithm of Dempster, Laird, and Rubin [1] for the maximization of likelihood functions. 3) This iterative method is used for maximum-likelihood estimation of image parameters in emission tomography and gammaray astronomy. We demonstrate that unconstrained likelihood estimation of image intensities from finite data sets yields unstable estimates. We show how Grenander's method of sieves can be used with the iterative algorithm to remove the instability. A bandwidth sieve is introduced resulting in an estimator which is smoothed via exponential splines. 4) We also derive a recursive algorithm for the generation of Toeplitz constrained maximum-likelihood estimators which at each iteration evaluates conditional mean estimates of the lag products based on the previous estimate of the covariance, from which the updated Toeplitz covariance is generated. We prove that the sequence of Toeplitz estimators has the property that they increase in likelihood, remain in the set of positive-definite Toeplitz covariances, and has all of its limit points stable and satisfying the necessary conditions for maximizing the likelihood.Keywords
This publication has 25 references indexed in Scilit:
- An Evaluation of the Use of Sieves for Producing Estimates. Of Radioactivity Distributions with the EM Algorithm for PETIEEE Transactions on Nuclear Science, 1986
- An Evaluation of an Improved Method for Computing Histograms in Dynamic Tracer Studies Using Positron-Emission TomographyIEEE Transactions on Nuclear Science, 1986
- The Use of Sieves to Stabilize Images Produced with the EM Algorithm for Emission TomographyIEEE Transactions on Nuclear Science, 1985
- Algorithms for removing recovery-related distortion from auditory-nerve discharge patternsThe Journal of the Acoustical Society of America, 1985
- A Statistical Model for Positron Emission TomographyJournal of the American Statistical Association, 1985
- Maximum‐likelihood estimation applied to electron microscopic autoradiographyJournal of Electron Microscopy Technique, 1985
- Maximum-Likelihood Reconstruction for Single-Photon Emission Computed-TomographyIEEE Transactions on Nuclear Science, 1985
- Maximum Likelihood PET with Real DataIEEE Transactions on Nuclear Science, 1984
- Maximum Likelihood Reconstruction for Emission TomographyIEEE Transactions on Medical Imaging, 1982
- Estimation of structured covariance matricesProceedings of the IEEE, 1982