On the Optimality of Conditional Expectation as a Bregman Predictor
Authors:
- Xin GuoXGXin Guo
- Hui WangHWHui Wang
- 27 June 2005
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 51 (7) , 2664-2669
Abstract
We consider the problem of predicting a random variable X from observations, denoted by a random variable Z. It is well known that the conditional expectation E[X|Z] is the optimal L/sup 2/ predictor (also known as "the least-mean-square error" predictor) of X, among all (Borel measurable) functions of Z. In this orrespondence, we provide necessary and sufficient conditions for the general loss functions under which the conditional expectation is the unique optimal predictor. We show that E[X|Z] is the optimal predictor for all Bregman loss functions (BLFs), of which the L/sup 2/ loss function is a special case. Moreover, under mild conditions, we show that the BLFs are exhaustive, i.e., if for every random variable X, the infimum of E[F(X,y)] over all constants y is attained by the expectation E[X], then F is a BLF.Keywords
This publication has 15 references indexed in Scilit:
- Clustering with Bregman Divergences2004
- Elliptic Partial Differential Equations of Second Order2001
- Asymptotic performance of vector quantizers with a perceptual distortion measureIEEE Transactions on Information Theory, 1999
- Convex Analysis and Variational Problems1999
- Generalized projections for non-negative functionsActa Mathematica Hungarica, 1995
- Why Least Squares and Maximum Entropy? An Axiomatic Approach to Inference for Linear Inverse ProblemsThe Annals of Statistics, 1991
- Probability with Martingales1991
- Entropic meansJournal of Mathematical Analysis and Applications, 1989
- Convex Analysis1970
- The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programmingUSSR Computational Mathematics and Mathematical Physics, 1967