Computing f‐divergences and distances of high‐dimensional probability density functions
Open Access
- 6 September 2022
- journal article
- research article
- Published by Wiley in Numerical Linear Algebra with Applications
- Vol. 30 (3)
Abstract
Very often, in the course of uncertainty quantification tasks or data analysis, one has to deal with high‐dimensional random variables. Here the interest is mainly to compute characterizations like the entropy, the Kullback–Leibler divergence, more general ‐divergences, or other such characteristics based on the probability density. The density is often not available directly, and it is a computational challenge to just represent it in a numerically feasible fashion in case the dimension is even moderately large. It is an even stronger numerical challenge to then actually compute said characteristics in the high‐dimensional case. In this regard it is proposed to approximate the discretized density in a compressed form, in particular by a low‐rank tensor. This can alternatively be obtained from the corresponding probability characteristic function, or more general representations of the underlying random variable. The mentioned characterizations need point‐wise functions like the logarithm. This normally rather trivial task becomes computationally difficult when the density is approximated in a compressed resp. low‐rank tensor format, as the point values are not directly accessible. The computations become possible by considering the compressed data as an element of an associative, commutative algebra with an inner product, and using matrix algorithms to accomplish the mentioned tasks. The representation as a low‐rank element of a high order tensor space allows to reduce the computational complexity and storage cost from exponential in the dimension to almost linear.Keywords
Funding Information
- Alexander von Humboldt-Stiftung
- Deutsche Forschungsgemeinschaft
This publication has 72 references indexed in Scilit:
- Polynomial Chaos: Modeling, Estimation, and Approximation2017
- On a new family of high‐order iterative methods for the matrix pth rootNumerical Linear Algebra with Applications, 2015
- A literature survey of low‐rank tensor approximation techniquesGAMM-Mitteilungen, 2013
- Tensor Spaces and Numerical Tensor Calculus2012
- A scalable optimization approach for fitting canonical tensor decompositionsJournal of Chemometrics, 2011
- Cross approximation in tensor electron density computationsNumerical Linear Algebra with Applications, 2010
- Uncertainty Quantification with Stochastic Finite Elements2004
- Multi‐Way Analysis with Applications in the Chemical Sciences2004
- Adaptive Blind Signal and Image Processing2002
- The Expression of a Tensor or a Polyadic as a Sum of ProductsJournal of Mathematics and Physics, 1927