A cautious user’s guide in applying HMMs to physical systems
- 2 December 2025
- journal article
- Published by AIP Publishing in The Journal of Chemical Physics
- Vol. 163 (21)
Abstract
Nature, as far as we know, evolves continuously through space and time. Yet, the ubiquitous hidden Markov model (HMM)—originally developed for discrete time and space analysis in natural language processing—remains a central tool in interpreting time series data drawn from physical systems. This raises a fundamental question: What are the implications of applying a discrete-state, discrete-time framework to analyze data generated by a continuously evolving system? Through synthetic data generated using Langevin dynamics in an effective potential, we explore under what circumstances HMMs yield interpretable results. Our analysis reveals that the discrete-state approximation acts primarily as an abstraction. As such, the inferred states visited in time often more closely reflecting the measurement protocol and modeling choices than features of the underlying physical potential. Crucially, we demonstrate that the states visited over the course of a time series recovered by the HMM can be tuned a priori by adjusting the data acquisition scheme. In fact, it is even possible to misleadingly and reproducibly recover “intermediate” states using different HMM tools for a system evolving in a single well potential. We conclude with a note of measured caution: while HMMs offer a mathematically elegant framework for time series inference, their use in physical modeling should be guided by an awareness of their limitations. In this light, we outline important generalizations of the HMM to continuous space and time and highlight the importance of a well calibrated measurement noise model.Keywords
Funding Information
- National Institute of General Medical Sciences (R01GM134426, R01GM130745)
- BRAIN Initiative (RF1MH128867)
- National Institute of General Medical Sciences (R35GM148237)
- Life Sciences Division, Army Research Office (ARO W911NF-23-1-0304)
This publication has 47 references indexed in Scilit:
- Deep-LASI, single-molecule data analysis softwareBiophysical Journal, 2024
- Learning continuous potentials from smFRETBiophysical Journal, 2022
- Generalizing HMMs to Continuous Time for Fast Kinetics: Hidden Markov Jump ProcessesBiophysical Journal, 2021
- A Systematic Review of Hidden Markov Models and Their ApplicationsArchives of Computational Methods in Engineering, 2020
- An Introduction to Infinite HMMs for Single-Molecule Data AnalysisBiophysical Journal, 2017
- ICON: An Adaptation of Infinite HMMs for Time Traces with DriftBiophysical Journal, 2017
- Analyzing Single-Molecule Time Series via Nonparametric Bayesian InferenceBiophysical Journal, 2015
- Learning Rates and States from Biophysical Time Series: A Bayesian Approach to Model Selection and Single-Molecule FRET DataBiophysical Journal, 2009
- Information Theory and an Extension of the Maximum Likelihood Principle1998
- Hidden Markov Models in Computational BiologyJournal of Molecular Biology, 1994