Erik M. Bollt
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Erik M. Bollt has a total of 112 co-authors. Here are the top co-authors by number of shared publications:
- Jie Sun
Theory Lab, Central Research Institute, 2012 Labs, Huawei Technologies Co. Ltd., Hong Kong SAR, China
14 shared publications - Jeremie Fish
Department of Ecology and Evolutionary Biology Yale University 06520 New Haven CT USA
7 shared publications - Joseph D. Skufca
Department of Mathematics, Clarkson University, Potsdam, NY 13699, USA
7 shared publications - Maurizio Porfiri
New York University Tandon School of Engineering
6 shared publications - Abd AlRahman R. AlMomani
Clarkson University
5 shared publications - Takashi Nishikawa
Department of Physics and Astronomy, Northwestern University, Evanston, USA
5 shared publications
Latest Publications
- Open Access
CausationEntropy: Pythonic Optimal Causation Entropy
Optimal Causation Entropy (oCSE) is a robust causal network modeling technique that reveals causal networks from dynamical systems and coupled oscillators, distinguishing direct from indirect paths. CausationEntropy is a Python package that implements oCSE and several of its significant optimizations and methodological extensions. In this paper, we introduce the version 1.1 release of CausationEntropy, which includes new synthetic data generators, plotting tools, and several advanced information-theoretical causal network discovery algorithms with criteria for estimating Gaussian, k-nearest neighbors (kNN), geometric k-nearest neighbors (geometric-kNN), kernel density (KDE) and Poisson entropic estimators. The package is easy to install from the PyPi software repository, is thoroughly documented, supplemented with extensive code examples, and is modularly structured to support future additions. The entire codebase is released under the MIT license and is available on GitHub and through PyPi Repository. We expect this package to serve as a benchmark tool for causal discovery in complex dynamical systems.
Open AccessCausationEntropy: Pythonic Optimal Causation Entropy
Optimal Causation Entropy (oCSE) is a robust causal network modeling technique that reveals causal networks from dynamical systems and coupled oscillators, distinguishing direct from indirect paths. CausationEntropy is a Python package that implements oCSE and several of its significant optimizations and methodological extensions. In this paper, we introduce the version 1.1 release of CausationEntropy, which includes new synthetic data generators, plotting tools, and several advanced information-theoretical causal network discovery algorithms with criteria for estimating Gaussian, k-nearest neighbors (kNN), geometric k-nearest neighbors (geometric-kNN), kernel density (KDE) and Poisson entropic estimators. The package is easy to install from the PyPi software repository, is thoroughly documented, supplemented with extensive code examples, and is modularly structured to support future additions. The entire codebase is released under the MIT license and is available on GitHub and through PyPi Repository. We expect this package to serve as a benchmark tool for causal discovery in complex dynamical systems.
- 1 December 2025
- Published by AIP Publishing in Chaos: An Interdisciplinary Journal of Nonlinear Science
- Vol. 35 (12)
On the emergence of numerical instabilities in next generation reservoir computing
Next Generation Reservoir Computing (NGRC) is a low-cost machine learning method for forecasting chaotic time series from data. Computational efficiency is crucial for scalable reservoir computing, requiring better strategies to reduce training cost. In this work, we uncover a connection between the numerical conditioning of the NGRC feature matrix—formed by polynomial evaluations on time-delay coordinates—and the long-term NGRC dynamics. We show that NGRC can be trained without regularization, reducing computational time. Our contributions are twofold. First, merging tools from numerical linear algebra and ergodic theory of dynamical systems, we systematically study how the feature matrix conditioning varies across hyperparameters. We demonstrate that the NGRC feature matrix tends to be ill-conditioned for short time lags, high-degree polynomials, and short length of training data. Second, we evaluate the impact of different numerical algorithms [Cholesky, singular value decomposition (SVD), and lower-upper decomposition] for solving the regularized least squares problem. Our results reveal that SVD-based training achieves accurate forecasts without regularization, being preferable when compared against the other algorithms.
- 1 December 2025
- Published by AIP Publishing in Chaos: An Interdisciplinary Journal of Nonlinear Science
- Vol. 35 (12)
On the emergence of numerical instabilities in next generation reservoir computing
Next Generation Reservoir Computing (NGRC) is a low-cost machine learning method for forecasting chaotic time series from data. Computational efficiency is crucial for scalable reservoir computing, requiring better strategies to reduce training cost. In this work, we uncover a connection between the numerical conditioning of the NGRC feature matrix—formed by polynomial evaluations on time-delay coordinates—and the long-term NGRC dynamics. We show that NGRC can be trained without regularization, reducing computational time. Our contributions are twofold. First, merging tools from numerical linear algebra and ergodic theory of dynamical systems, we systematically study how the feature matrix conditioning varies across hyperparameters. We demonstrate that the NGRC feature matrix tends to be ill-conditioned for short time lags, high-degree polynomials, and short length of training data. Second, we evaluate the impact of different numerical algorithms [Cholesky, singular value decomposition (SVD), and lower-upper decomposition] for solving the regularized least squares problem. Our results reveal that SVD-based training achieves accurate forecasts without regularization, being preferable when compared against the other algorithms.
- Open Access
- 25 November 2025
- Published by Springer Nature in Scientific Reports
- Vol. 15 (1) , 41967
Tree-based learning for high-fidelity prediction of chaos
Forecasting chaotic systems using machine learning has become highly sought after due to its potential applications in predicting climate and weather phenomena, stock market indices, and pathological activity in biomedical signals. However, existing solutions, such as neural network-based reservoir computing (RC) and long-short-term memory (LSTM), contain numerous model hyperparameters that must be tuned, often requiring high computational resources and large training datasets. Here, we propose a computationally simpler regression tree ensemble-based technique to predict the temporal evolution of chaotic systems in data-driven environments. Furthermore, we introduce a heuristic procedure to prescribe hyperparameters through automated statistical analysis of training data, which eliminates the need for the user to perform hyperparameter tuning. We investigate the efficacy of the proposed hyperparameter prescription procedure through numerical experiments. Lastly, we demonstrate the state-of-the-art performance of our proposed approach on several benchmark tasks, including the Southern Oscillation Index, a crucial but noisy climate time series with limited samples, to illustrate its effectiveness in real-world settings.
- 25 November 2025
- Published by Springer Nature in Scientific Reports
- Vol. 15 (1) , 41967
Open AccessTree-based learning for high-fidelity prediction of chaos
Forecasting chaotic systems using machine learning has become highly sought after due to its potential applications in predicting climate and weather phenomena, stock market indices, and pathological activity in biomedical signals. However, existing solutions, such as neural network-based reservoir computing (RC) and long-short-term memory (LSTM), contain numerous model hyperparameters that must be tuned, often requiring high computational resources and large training datasets. Here, we propose a computationally simpler regression tree ensemble-based technique to predict the temporal evolution of chaotic systems in data-driven environments. Furthermore, we introduce a heuristic procedure to prescribe hyperparameters through automated statistical analysis of training data, which eliminates the need for the user to perform hyperparameter tuning. We investigate the efficacy of the proposed hyperparameter prescription procedure through numerical experiments. Lastly, we demonstrate the state-of-the-art performance of our proposed approach on several benchmark tasks, including the Southern Oscillation Index, a crucial but noisy climate time series with limited samples, to illustrate its effectiveness in real-world settings.
- 3 October 2025
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Applied Dynamical Systems
- Vol. 24 (4) , 2489-2523
Globally Optimal Stretching Foliations of Dynamical Systems
There is a well-established understanding of maximizing and minimizing local stretching associated with a map; this can be couched in terms of an eigenproblem associated with the Cauchy–Green tensor. The local eigenvalues correspond to stretching rates, while the eigenvector directions identify maximal and minimal stretching directions. Generating hyperstreamlines of these eigenvector fields has been popular; however various claims that these collections of curves correspond to global optimizers of stretching remain unsubstantiated. In this paper, a formulation in terms of restricted foliations is presented, for which these curves form a well-defined solution as the most and least unstable objects, with respect to a global definition. The theory works for any map in [Formula: see text]-dimensions, derived, for example, from repeated applications of a discrete dynamical system, or from a finite-time flow of a continuous one. Further insight is obtained into the two-dimensional situation via an elementary approach, including a simple singularity classification criterion and an understanding of numerical artifacts in foliation computations (e.g., special curves along which hyperstreamlines veer and/or do not cross). Illustrations and validations of the results to the Hénon map, the double-gyre flow, and the standard (Chirikov) map are provided.
- 3 October 2025
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Applied Dynamical Systems
- Vol. 24 (4) , 2489-2523
Globally Optimal Stretching Foliations of Dynamical Systems
There is a well-established understanding of maximizing and minimizing local stretching associated with a map; this can be couched in terms of an eigenproblem associated with the Cauchy–Green tensor. The local eigenvalues correspond to stretching rates, while the eigenvector directions identify maximal and minimal stretching directions. Generating hyperstreamlines of these eigenvector fields has been popular; however various claims that these collections of curves correspond to global optimizers of stretching remain unsubstantiated. In this paper, a formulation in terms of restricted foliations is presented, for which these curves form a well-defined solution as the most and least unstable objects, with respect to a global definition. The theory works for any map in [Formula: see text]-dimensions, derived, for example, from repeated applications of a discrete dynamical system, or from a finite-time flow of a continuous one. Further insight is obtained into the two-dimensional situation via an elementary approach, including a simple singularity classification criterion and an understanding of numerical artifacts in foliation computations (e.g., special curves along which hyperstreamlines veer and/or do not cross). Illustrations and validations of the results to the Hénon map, the double-gyre flow, and the standard (Chirikov) map are provided.
- 1 July 2025
- Published by AIP Publishing in Chaos: An Interdisciplinary Journal of Nonlinear Science
- Vol. 35 (7)
Locality blended next-generation reservoir computing for attention accuracy
We extend an advanced variation of a machine learning algorithm, next-generation reservoir computing (NGRC), to forecast synthetic data generated by the Ikeda map, which is a model of a nonlinear optical cavity with an injected laser beam. The machine learning model is created by observing time-series data generated by the Ikeda map during a training phase, and the trained model is used to forecast the behavior in a closed-loop mode where it is no longer supplied with data from the map. The Ikeda map is a particularly challenging problem to learn because of the complicated map functions. We overcome the challenge by a novel improvement of the NGRC concept by emphasizing simpler polynomial models localized to well-designed regions of phase space and then blending these models between regions, a method that we call locality blended next-generation reservoir computing. This approach allows for better performance with relatively smaller data sets in comparison to deep learning methods and gives a new level of interpretability. We achieve forecasting horizons exceeding five Lyapunov times, and we demonstrate that the model reproduces the invariant measure of the attractor beyond the short-term forecasting horizon.
- 1 July 2025
- Published by AIP Publishing in Chaos: An Interdisciplinary Journal of Nonlinear Science
- Vol. 35 (7)
Locality blended next-generation reservoir computing for attention accuracy
We extend an advanced variation of a machine learning algorithm, next-generation reservoir computing (NGRC), to forecast synthetic data generated by the Ikeda map, which is a model of a nonlinear optical cavity with an injected laser beam. The machine learning model is created by observing time-series data generated by the Ikeda map during a training phase, and the trained model is used to forecast the behavior in a closed-loop mode where it is no longer supplied with data from the map. The Ikeda map is a particularly challenging problem to learn because of the complicated map functions. We overcome the challenge by a novel improvement of the NGRC concept by emphasizing simpler polynomial models localized to well-designed regions of phase space and then blending these models between regions, a method that we call locality blended next-generation reservoir computing. This approach allows for better performance with relatively smaller data sets in comparison to deep learning methods and gives a new level of interpretability. We achieve forecasting horizons exceeding five Lyapunov times, and we demonstrate that the model reproduces the invariant measure of the attractor beyond the short-term forecasting horizon.
- Top Cited PapersOpen Access
- 10 April 2025
- Published by American Physical Society (APS) in Physical Review Research
- Vol. 7 (2)
Data-driven model discovery with Kolmogorov-Arnold networks
Data-driven model discovery of complex dynamical systems can be done using sparse optimization, but it has a fundamental limitation: sparsity in that the underlying governing equations of the system contain only a small number of elementary mathematical functions. Examples where sparse optimization fails abound, including the classic Ikeda or optical-cavity map in nonlinear dynamics, as well as a wide variety of ecosystems. Another approach is based on machine learning, e.g., deep neural networks, which excels at capturing system behavior from data but functions as black boxes, offering little insight into how inputs influence the outputs. We propose a general model-discovery framework based on the Kolmogorov-Arnold networks (KANs) that are not constrained by the sparsity condition. The KAN framework with a simple structure is capable of accurately capturing the complex behavior of dynamical systems that do not meet the sparsity requirement while offering greater interpretability compared to conventional neural networks. This interpretability provides insights into the dynamics generating the data, which are typically inaccessible in traditional black-box function approximation methods. We demonstrate nonuniqueness in that a large number of approximate models of the system can be found that generate the same invariant set with the true statistics such as the Lyapunov exponents and Kullback–Leibler divergence. An analogy to shadowing of numerical trajectories in chaotic systems is pointed out.
- 10 April 2025
- Published by American Physical Society (APS) in Physical Review Research
- Vol. 7 (2)
Top Cited PapersOpen AccessData-driven model discovery with Kolmogorov-Arnold networks
Data-driven model discovery of complex dynamical systems can be done using sparse optimization, but it has a fundamental limitation: sparsity in that the underlying governing equations of the system contain only a small number of elementary mathematical functions. Examples where sparse optimization fails abound, including the classic Ikeda or optical-cavity map in nonlinear dynamics, as well as a wide variety of ecosystems. Another approach is based on machine learning, e.g., deep neural networks, which excels at capturing system behavior from data but functions as black boxes, offering little insight into how inputs influence the outputs. We propose a general model-discovery framework based on the Kolmogorov-Arnold networks (KANs) that are not constrained by the sparsity condition. The KAN framework with a simple structure is capable of accurately capturing the complex behavior of dynamical systems that do not meet the sparsity requirement while offering greater interpretability compared to conventional neural networks. This interpretability provides insights into the dynamics generating the data, which are typically inaccessible in traditional black-box function approximation methods. We demonstrate nonuniqueness in that a large number of approximate models of the system can be found that generate the same invariant set with the true statistics such as the Lyapunov exponents and Kullback–Leibler divergence. An analogy to shadowing of numerical trajectories in chaotic systems is pointed out.
- Open Access
The problem of infinite information flow
We study conditional mutual information (cMI) between a pair of variables $X,Y$ given a third one $Z$ and derived quantities including transfer entropy (TE) and causation entropy (CE) in the dynamically relevant context where $X=T(Y,Z)$ is determined by $Y,Z$ via a deterministic transformation $T$. Under mild continuity assumptions on their distributions, we prove a zero-infinity dichotomy for cMI for a wide class of $T$, which gives a yes-or-no answer to the question of information flow as quantified by TE or CE. Such an answer fails to distinguish between the relative amounts of information flow. To resolve this problem, we propose a discretization strategy and a conjectured formula to discern the \textit{relative ambiguities} of the system, which can serve as a reliable proxy for the relative amounts of information flow. We illustrate and validate this approach with numerical evidence.Open AccessThe problem of infinite information flow
We study conditional mutual information (cMI) between a pair of variables $X,Y$ given a third one $Z$ and derived quantities including transfer entropy (TE) and causation entropy (CE) in the dynamically relevant context where $X=T(Y,Z)$ is determined by $Y,Z$ via a deterministic transformation $T$. Under mild continuity assumptions on their distributions, we prove a zero-infinity dichotomy for cMI for a wide class of $T$, which gives a yes-or-no answer to the question of information flow as quantified by TE or CE. Such an answer fails to distinguish between the relative amounts of information flow. To resolve this problem, we propose a discretization strategy and a conjectured formula to discern the \textit{relative ambiguities} of the system, which can serve as a reliable proxy for the relative amounts of information flow. We illustrate and validate this approach with numerical evidence. - 1 December 2024
- Published by AIP Publishing in Chaos: An Interdisciplinary Journal of Nonlinear Science
- Vol. 34 (12)
How neural networks work: Unraveling the mystery of randomized neural networks for functions and chaotic dynamical systems
Artificial Neural Networks (ANNs) have proven to be fantastic at a wide range of machine learning tasks, and they have certainly come into their own in all sorts of technologies that are widely consumed today in society as a whole. A basic task of machine learning that neural networks are well suited to is supervised learning, including when learning orbits from time samples of dynamical systems. The usual construct in ANN is to fully train all of the perhaps many millions of parameters that define the network architecture. However, there are certain ANN algorithms that work well with random designs. We have previously presented an explanation as to how the reservoir computing recurrent neural network architecture succeeds despite randomness. Here, we explain how the random feedforward neural networks called the random project networks work. In particular, we present examples for both general function learning and also for learning a flow from samples of orbits of chaotic dynamical systems. There is an interesting geometric explanation of the success, in the case of the ReLu activation function, that relates to the classical mathematical question of how configurations of random lines fall in a plane, or how planes or hyperplanes may fall in higher dimensional spaces. These random configurations lead to a refinement of the domain so that piecewise linear continuous functions result that are dense in continuous functions. This relates neural networks to finite element methods. We highlight the universality of this representation by forecasting the skill of chaotic dynamical systems.
- 1 December 2024
- Published by AIP Publishing in Chaos: An Interdisciplinary Journal of Nonlinear Science
- Vol. 34 (12)
How neural networks work: Unraveling the mystery of randomized neural networks for functions and chaotic dynamical systems
Artificial Neural Networks (ANNs) have proven to be fantastic at a wide range of machine learning tasks, and they have certainly come into their own in all sorts of technologies that are widely consumed today in society as a whole. A basic task of machine learning that neural networks are well suited to is supervised learning, including when learning orbits from time samples of dynamical systems. The usual construct in ANN is to fully train all of the perhaps many millions of parameters that define the network architecture. However, there are certain ANN algorithms that work well with random designs. We have previously presented an explanation as to how the reservoir computing recurrent neural network architecture succeeds despite randomness. Here, we explain how the random feedforward neural networks called the random project networks work. In particular, we present examples for both general function learning and also for learning a flow from samples of orbits of chaotic dynamical systems. There is an interesting geometric explanation of the success, in the case of the ReLu activation function, that relates to the classical mathematical question of how configurations of random lines fall in a plane, or how planes or hyperplanes may fall in higher dimensional spaces. These random configurations lead to a refinement of the domain so that piecewise linear continuous functions result that are dense in continuous functions. This relates neural networks to finite element methods. We highlight the universality of this representation by forecasting the skill of chaotic dynamical systems.
- Open Access
Fractal Conditional Correlation Dimension Infers Complex Causal Networks
Determining causal inference has become popular in physical and engineering applications. While the problem has immense challenges, it provides a way to model the complex networks by observing the time series. In this paper, we present the optimal conditional correlation dimensional geometric information flow principle ($oGeoC$) that can reveal direct and indirect causal relations in a network through geometric interpretations. We introduce two algorithms that utilize the $oGeoC$ principle to discover the direct links and then remove indirect links. The algorithms are evaluated using coupled logistic networks. The results indicate that when the number of observations is sufficient, the proposed algorithms are highly accurate in identifying direct causal links and have a low false positive rate.Open AccessFractal Conditional Correlation Dimension Infers Complex Causal Networks
Determining causal inference has become popular in physical and engineering applications. While the problem has immense challenges, it provides a way to model the complex networks by observing the time series. In this paper, we present the optimal conditional correlation dimensional geometric information flow principle ($oGeoC$) that can reveal direct and indirect causal relations in a network through geometric interpretations. We introduce two algorithms that utilize the $oGeoC$ principle to discover the direct links and then remove indirect links. The algorithms are evaluated using coupled logistic networks. The results indicate that when the number of observations is sufficient, the proposed algorithms are highly accurate in identifying direct causal links and have a low false positive rate. - Open Access
Fractal Conditional Correlation Dimension Infers Complex Causal Networks
Determining causal inference has become popular in physical and engineering applications. While the problem has immense challenges, it provides a way to model the complex networks by observing the time series. In this paper, we present the optimal conditional correlation dimensional geometric information flow principle () that can reveal direct and indirect causal relations in a network through geometric interpretations. We introduce two algorithms that utilize the principle to discover the direct links and then remove indirect links. The algorithms are evaluated using coupled logistic networks. The results indicate that when the number of observations is sufficient, the proposed algorithms are highly accurate in identifying direct causal links and have a low false positive rate.Open AccessFractal Conditional Correlation Dimension Infers Complex Causal Networks
Determining causal inference has become popular in physical and engineering applications. While the problem has immense challenges, it provides a way to model the complex networks by observing the time series. In this paper, we present the optimal conditional correlation dimensional geometric information flow principle () that can reveal direct and indirect causal relations in a network through geometric interpretations. We introduce two algorithms that utilize the principle to discover the direct links and then remove indirect links. The algorithms are evaluated using coupled logistic networks. The results indicate that when the number of observations is sufficient, the proposed algorithms are highly accurate in identifying direct causal links and have a low false positive rate.