Parameter estimation for Gibbs distributions
Preprint
Abstract
We consider Gibbs distributions, which are families of probability distributions over a discrete space $\Omega$ with probability mass function of the form $\mu^\Omega_\beta(\omega) \propto e^{\beta H(\omega)}$ for $\beta$ in an interval $[\beta_{\min}, \beta_{\max}]$ and $H(\omega) \in \{0 \} \cup [1, n]$. The \emph{partition function} is the normalization factor $Z(\beta)=\sum_{\omega\in\Omega}e^{\beta H(\omega)}$. Two important parameters of these distributions are the partition ratio $q = \log \tfrac{Z(\beta_{\max})}{Z(\beta_{\min})}$ and the counts $c_x = |H^{-1}(x)|$. These are correlated with system parameters in a number of physical applications and sampling algorithms. Our first main result is to estimate the values $c_x$ using roughly $\tilde O( \frac{q}{\varepsilon^2})$ samples for general Gibbs distributions and $\tilde O( \frac{n^2}{\varepsilon^2} )$ samples for integer-valued distributions (ignoring some second-order terms and parameters), and we show this is optimal up to logarithmic factors. We illustrate with improved algorithms for counting connected subgraphs and perfect matchings in a graph. A key subroutine we develop is to estimate the partition function $Z$; specifically, we generate a data structure capable of estimating $Z(\beta)$ for all values $\beta$, without further samples. Constructing the data structure requires $\tilde O(\frac{q}{\varepsilon^2})$ samples for general Gibbs distributions and $\tilde O(\frac{n^2}{\varepsilon^2})$ samples for integer-valued distributions. This improves over a prior algorithm of Kolmogorov (2018) which computes the single point estimate $Z(\beta_{\max})$ using $\tilde O(\frac{q}{\varepsilon^2})$ samples. We show matching lower bounds, demonstrating that this complexity is optimal as a function of $n$ and $q$ up to logarithmic terms.
Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Tight bounds for popping algorithmsRandom Structures & Algorithms, 2020
- A Polynomial-Time Approximation Algorithm for All-Terminal Network ReliabilitySIAM Journal on Computing, 2019
- Using TPA for Bayesian Inference*2011
- Adaptive simulated annealing: A near-optimal connection between sampling and countingJournal of the ACM, 2009
- Accelerating Simulated Annealing for the Permanent and Combinatorial Counting ProblemsSIAM Journal on Computing, 2008
- Wang-Landau algorithm: A theoretical analysis of the saturation of the errorThe Journal of Chemical Physics, 2007
- Efficient, Multiple-Range Random Walk Algorithm to Calculate the Density of StatesPhysical Review Letters, 2001
- Upper and lower bounds for sums of random variablesInsurance: Mathematics and Economics, 2000
- Stop-loss order for portfolios of dependent risksInsurance: Mathematics and Economics, 1997
- Approximating the PermanentSIAM Journal on Computing, 1989