Hessian and Concavity of Mutual Information, Differential Entropy, and Entropy Power in Linear Vector Gaussian Channels
Authors:
- 14 July 2009
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 55 (8) , 3613-3628
Abstract
Within the framework of linear vector Gaussian channels with arbitrary signaling, the Jacobian of the minimum mean square error and Fisher information matrices with respect to arbitrary parameters of the system are calculated in this paper. Capitalizing on prior research where the minimum mean square error and Fisher information matrices were linked to information-theoretic quantities through differentiation, the Hessian of the mutual information and the entropy are derived. These expressions are then used to assess the concavity properties of mutual information and entropy under different channel conditions and also to derive a multivariate version of an entropy power inequality due to Costa.Keywords
This publication has 26 references indexed in Scilit:
- On Mutual Information, Likelihood Ratios, and Estimation Error for the Additive Gaussian ChannelIEEE Transactions on Information Theory, 2005
- Mutual Information and Minimum Mean-Square Error in Gaussian ChannelsIEEE Transactions on Information Theory, 2005
- Second-Order Asymptotics of Mutual InformationIEEE Transactions on Information Theory, 2004
- Joint tx-rx beamforming design for multicarrier mimo channels: a unified framework for convex optimizationIEEE Transactions on Signal Processing, 2003
- A short proof of the "concavity of entropy power"IEEE Transactions on Information Theory, 2000
- Information theoretic inequalitiesIEEE Transactions on Information Theory, 1991
- Simple proof of the concavity of the entropy power with respect to added Gaussian noiseIEEE Transactions on Information Theory, 1989
- A new entropy power inequalityIEEE Transactions on Information Theory, 1985
- Mutual information of the white Gaussian channel with and without feedbackIEEE Transactions on Information Theory, 1971
- Some inequalities satisfied by the quantities of information of Fisher and ShannonInformation and Control, 1959