Tyrus Berry
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Tyrus Berry has a total of 43 co-authors. Here are the top co-authors by number of shared publications:
- Timothy Sauer
George Mason University
10 shared publications - Donald Ebeigbe
Department of Electrical Engineering, The Pennsylvania State University, University Park, PA, USA
5 shared publications - Steven J. Schiff
National Institutes of Health, 9000 Rockville Pike, 31 Center Drive MSC 2220, Bethesda, Maryland, USA
5 shared publications - Andrew John Whalen
Yale University
4 shared publications - Andrew Draganov
Department of Computer Science Aarhus Uni- versity Aarhus Denmark
3 shared publications - Davide Mottin
Aarhus University, Aarhus, Denmark
3 shared publications
Latest Publications
- 27 May 2025
- Published by American Physical Society (APS) in Physical Review E
- Vol. 111 (5) , 054135
Generalized unscented transformation for forecasting non-Gaussian processes
The observations of linear and nonlinear physical processes are subject to random errors, which can be represented by a wide variety of probability distributions. In contrast, most estimation and inference techniques rely on a Gaussian assumption, which may limit our ability to make model-based predictions. There is a need for data assimilation methods that can capture and leverage the higher moments of these physical processes for state estimation and forecasting. In this paper, we develop the generalized unscented transform (GenUT), which uses a minimal number of sample points to accurately capture elements of the higher moments of most probability distributions. Constraints can be analytically enforced on the sample points while guaranteeing at least second-order accuracy. The GenUT is widely applicable to non-Gaussian distributions, which can substantially improve the assimilation of observations of nonlinear physics, such as the modeling of infectious diseases.
- 27 May 2025
- Published by American Physical Society (APS) in Physical Review E
- Vol. 111 (5) , 054135
Generalized unscented transformation for forecasting non-Gaussian processes
The observations of linear and nonlinear physical processes are subject to random errors, which can be represented by a wide variety of probability distributions. In contrast, most estimation and inference techniques rely on a Gaussian assumption, which may limit our ability to make model-based predictions. There is a need for data assimilation methods that can capture and leverage the higher moments of these physical processes for state estimation and forecasting. In this paper, we develop the generalized unscented transform (GenUT), which uses a minimal number of sample points to accurately capture elements of the higher moments of most probability distributions. Constraints can be analytically enforced on the sample points while guaranteeing at least second-order accuracy. The GenUT is widely applicable to non-Gaussian distributions, which can substantially improve the assimilation of observations of nonlinear physics, such as the modeling of infectious diseases.
- 20 February 2025
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Applied Dynamical Systems
- Vol. 24 (1) , 633-654
Reconstruction of Network Dynamics from Partial Observations
We investigate the reconstruction of time series from dynamical networks that are partially observed. In particular, we address the extent to which the time series at a node of the network can be successfully reconstructed when measuring from another node, or subset of nodes, corrupted by observational noise. We will assume the dynamical equations of the network are known, and that the dynamics are not necessarily low-dimensional. The case of linear dynamics is treated first and leads to a definition of observation error magnification factor (OEMF) that measures the magnification of noise in the reconstruction process. Subsequently, the definition is applied to nonlinear and chaotic dynamics. Comparison of OEMF for different target/observer combinations can lead to better understanding of how to optimally observe a network. As part of the study, a computational method for reconstructing time series from partial observations is presented and analyzed.
- 20 February 2025
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Applied Dynamical Systems
- Vol. 24 (1) , 633-654
Reconstruction of Network Dynamics from Partial Observations
We investigate the reconstruction of time series from dynamical networks that are partially observed. In particular, we address the extent to which the time series at a node of the network can be successfully reconstructed when measuring from another node, or subset of nodes, corrupted by observational noise. We will assume the dynamical equations of the network are known, and that the dynamics are not necessarily low-dimensional. The case of linear dynamics is treated first and leads to a definition of observation error magnification factor (OEMF) that measures the magnification of noise in the reconstruction process. Subsequently, the definition is applied to nonlinear and chaotic dynamics. Comparison of OEMF for different target/observer combinations can lead to better understanding of how to optimally observe a network. As part of the study, a computational method for reconstructing time series from partial observations is presented and analyzed.
- 6 February 2025
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Review
- Vol. 67 (1) , 107-137
Limits of Learning Dynamical Systems
A dynamical system is a transformation of a phase space, and the transformation law is the primary means of defining as well as identifying the dynamical system and is the object of focus of many learning techniques. However, there are many secondary aspects of dynamical systems—invariant sets, the Koopman operator, and Markov approximations—that provide alternative objectives for learning techniques. Crucially, while many learning methods are focused on the transformation law, we find that forecast performance can depend on how well these other aspects of the dynamics are approximated. These different facets of a dynamical system correspond to objects in completely different spaces—namely, interpolation spaces, compact Hausdorff sets, unitary operators, and Markov operators, respectively. Thus, learning techniques targeting any of these four facets perform different kinds of approximations. We examine whether an approximation of any one of these aspects of the dynamics could lead to an approximation of another facet. Many connections and obstructions are brought to light in this analysis. Special focus is placed on methods of learning the primary feature—the dynamics law itself. The main question considered is the connection between learning this law and reconstructing the Koopman operator and the invariant set. The answers are tied to the ergodic and topological properties of the dynamics, and they reveal how these properties determine the limits of forecasting techniques.
- 6 February 2025
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Review
- Vol. 67 (1) , 107-137
Limits of Learning Dynamical Systems
A dynamical system is a transformation of a phase space, and the transformation law is the primary means of defining as well as identifying the dynamical system and is the object of focus of many learning techniques. However, there are many secondary aspects of dynamical systems—invariant sets, the Koopman operator, and Markov approximations—that provide alternative objectives for learning techniques. Crucially, while many learning methods are focused on the transformation law, we find that forecast performance can depend on how well these other aspects of the dynamics are approximated. These different facets of a dynamical system correspond to objects in completely different spaces—namely, interpolation spaces, compact Hausdorff sets, unitary operators, and Markov operators, respectively. Thus, learning techniques targeting any of these four facets perform different kinds of approximations. We examine whether an approximation of any one of these aspects of the dynamics could lead to an approximation of another facet. Many connections and obstructions are brought to light in this analysis. Special focus is placed on methods of learning the primary feature—the dynamics law itself. The main question considered is the connection between learning this law and reconstructing the Koopman operator and the invariant set. The answers are tied to the ergodic and topological properties of the dynamics, and they reveal how these properties determine the limits of forecasting techniques.
- 27 January 2025
- Published by American Physical Society (APS) in Physical Review E
- Vol. 111 (1) , 014317
Existence and stability of equilibria in infectious disease dynamics with behavioral feedback
Mathematical models have provided a general framework for understanding the dynamics and control of infectious disease. Many compartmental models are limited in that they do not account for the range of behavioral feedbacks that have been observed in the response to emerging infections. Here we expand on the SIR compartmental model framework by introducing a general class of behavioral feedbacks that encompasses both individual responses and nonpharmaceutical interventions. By linking transmission dynamics and behavior, this class of models can capture the interplay of disease incidence, behavioral response, and controls such as vaccination. We prove mathematically the existence of two endemic equilibria depending on the vaccination rate: one in the presence of low vaccination but with reduced societal activity (the "new normal"), and one with return to normal activity but with vaccination rate below that required for disease elimination. Establishing the existence and stability of these equilibria is a precursor to designing control strategies that may exploit them.
- 27 January 2025
- Published by American Physical Society (APS) in Physical Review E
- Vol. 111 (1) , 014317
Existence and stability of equilibria in infectious disease dynamics with behavioral feedback
Mathematical models have provided a general framework for understanding the dynamics and control of infectious disease. Many compartmental models are limited in that they do not account for the range of behavioral feedbacks that have been observed in the response to emerging infections. Here we expand on the SIR compartmental model framework by introducing a general class of behavioral feedbacks that encompasses both individual responses and nonpharmaceutical interventions. By linking transmission dynamics and behavior, this class of models can capture the interplay of disease incidence, behavioral response, and controls such as vaccination. We prove mathematically the existence of two endemic equilibria depending on the vaccination rate: one in the presence of low vaccination but with reduced societal activity (the "new normal"), and one with return to normal activity but with vaccination rate below that required for disease elimination. Establishing the existence and stability of these equilibria is a precursor to designing control strategies that may exploit them.
- 9 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Control Systems Letters
- Vol. 8, 2865-2870
Nonlinear Kalman Filtering in the Absence of Direct Functional Relationships Between Measurement and State
This letter introduces a Kalman Filter framework for systems with process noise and measurements characterized by state-dependent, nonlinear conditional means and covariances. Estimating such general nonlinear models is challenging because traditional methods, such as the Extended Kalman Filter, linearize only functions – not noise – and require state-independent covariances. These limitations often necessitate Bayesian approaches that rely on specific distribution assumptions. To address these challenges, we propose a framework that employs a recursive least squares method that relies solely on conditional means and covariances, eliminating the need for explicit probability distributions. By applying first-order linearizations and incorporating targeted modifications to manage state dependence, the filter simplifies implementation, reduces computational demands, and provides a practical solution for systems that deviate from the assumptions underlying traditional Kalman filters. Simulation results on a compartmental model demonstrate performance comparable to sequential Monte Carlo methods while significantly lowering computational costs, effectively addressing real-world challenges of scalability and precision.
- 9 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Control Systems Letters
- Vol. 8, 2865-2870
Nonlinear Kalman Filtering in the Absence of Direct Functional Relationships Between Measurement and State
This letter introduces a Kalman Filter framework for systems with process noise and measurements characterized by state-dependent, nonlinear conditional means and covariances. Estimating such general nonlinear models is challenging because traditional methods, such as the Extended Kalman Filter, linearize only functions – not noise – and require state-independent covariances. These limitations often necessitate Bayesian approaches that rely on specific distribution assumptions. To address these challenges, we propose a framework that employs a recursive least squares method that relies solely on conditional means and covariances, eliminating the need for explicit probability distributions. By applying first-order linearizations and incorporating targeted modifications to manage state dependence, the filter simplifies implementation, reduces computational demands, and provides a practical solution for systems that deviate from the assumptions underlying traditional Kalman filters. Simulation results on a compartmental model demonstrate performance comparable to sequential Monte Carlo methods while significantly lowering computational costs, effectively addressing real-world challenges of scalability and precision.
- 15 July 2024
- Published by Springer Nature
- p. 275-296
Applications of Topological Graph Theory to 2-Manifold Learning
- 15 July 2024
- Published by Springer Nature
- p. 275-296
Applications of Topological Graph Theory to 2-Manifold Learning
- Open Access
- 14 February 2024
- Published by Frontiers Media SA in Frontiers in Computer Science
- Vol. 6, 1274181
On-manifold projected gradient descent
This study provides a computable, direct, and mathematically rigorous approximation to the differential geometry of class manifolds for high-dimensional data, along with non-linear projections from input space onto these class manifolds. The tools are applied to the setting of neural network image classifiers, where we generate novel, on-manifold data samples and implement a projected gradient descent algorithm for on-manifold adversarial training. The susceptibility of neural networks (NNs) to adversarial attack highlights the brittle nature of NN decision boundaries in input space. Introducing adversarial examples during training has been shown to reduce the susceptibility of NNs to adversarial attack; however, it has also been shown to reduce the accuracy of the classifier if the examples are not valid examples for that class. Realistic “on-manifold” examples have been previously generated from class manifolds in the latent space of an autoencoder. Our study explores these phenomena in a geometric and computational setting that is much closer to the raw, high-dimensional input space than what can be provided by VAE or other black box dimensionality reductions. We employ conformally invariant diffusion maps (CIDM) to approximate class manifolds in diffusion coordinates and develop the Nyström projection to project novel points onto class manifolds in this setting. On top of the manifold approximation, we leverage the spectral exterior calculus (SEC) to determine geometric quantities such as tangent vectors of the manifold. We use these tools to obtain adversarial examples that reside on a class manifold, yet fool a classifier. These misclassifications then become explainable in terms of human-understandable manipulations within the data, by expressing the on-manifold adversary in the semantic basis on the manifold.
- 14 February 2024
- Published by Frontiers Media SA in Frontiers in Computer Science
- Vol. 6, 1274181
Open AccessOn-manifold projected gradient descent
This study provides a computable, direct, and mathematically rigorous approximation to the differential geometry of class manifolds for high-dimensional data, along with non-linear projections from input space onto these class manifolds. The tools are applied to the setting of neural network image classifiers, where we generate novel, on-manifold data samples and implement a projected gradient descent algorithm for on-manifold adversarial training. The susceptibility of neural networks (NNs) to adversarial attack highlights the brittle nature of NN decision boundaries in input space. Introducing adversarial examples during training has been shown to reduce the susceptibility of NNs to adversarial attack; however, it has also been shown to reduce the accuracy of the classifier if the examples are not valid examples for that class. Realistic “on-manifold” examples have been previously generated from class manifolds in the latent space of an autoencoder. Our study explores these phenomena in a geometric and computational setting that is much closer to the raw, high-dimensional input space than what can be provided by VAE or other black box dimensionality reductions. We employ conformally invariant diffusion maps (CIDM) to approximate class manifolds in diffusion coordinates and develop the Nyström projection to project novel points onto class manifolds in this setting. On top of the manifold approximation, we leverage the spectral exterior calculus (SEC) to determine geometric quantities such as tangent vectors of the manifold. We use these tools to obtain adversarial examples that reside on a class manifold, yet fool a classifier. These misclassifications then become explainable in terms of human-understandable manipulations within the data, by expressing the on-manifold adversary in the semantic basis on the manifold.
- 1 January 2024
- Published by American Institute of Mathematical Sciences (AIMS) in Foundations of Data Science
- Vol. 6 (3) , 308-323
Convergence analysis of the rank-restricted soft SVD algorithm
The soft SVD is a robust matrix decomposition algorithm and a key component of matrix completion methods. However, computing the soft SVD for large sparse matrices is often impractical using conventional numerical methods for the SVD due to large memory requirements. The Rank-Restricted Soft SVD (RRSS) algorithm introduced by Hastie et al. addressed this issue by sequentially computing low-rank SVDs that easily fit in memory. We analyze the convergence of the standard RRSS algorithm and we give examples where the standard algorithm does not converge. We show that convergence requires a modification of the standard algorithm, and is related to non-uniqueness of the SVD. Our modification specifies a consistent choice of sign for the left singular vectors of the low-rank SVDs in the iteration. Under these conditions, we prove linear convergence of the singular vectors using a technique motivated by alternating subspace iteration. We then derive a fixed point iteration for the evolution of the singular values and show linear convergence to the soft thresholded singular values of the original matrix. This last step requires a perturbation result for fixed point iterations which may be of independent interest.
- 1 January 2024
- Published by American Institute of Mathematical Sciences (AIMS) in Foundations of Data Science
- Vol. 6 (3) , 308-323
Convergence analysis of the rank-restricted soft SVD algorithm
The soft SVD is a robust matrix decomposition algorithm and a key component of matrix completion methods. However, computing the soft SVD for large sparse matrices is often impractical using conventional numerical methods for the SVD due to large memory requirements. The Rank-Restricted Soft SVD (RRSS) algorithm introduced by Hastie et al. addressed this issue by sequentially computing low-rank SVDs that easily fit in memory. We analyze the convergence of the standard RRSS algorithm and we give examples where the standard algorithm does not converge. We show that convergence requires a modification of the standard algorithm, and is related to non-uniqueness of the SVD. Our modification specifies a consistent choice of sign for the left singular vectors of the low-rank SVDs in the iteration. Under these conditions, we prove linear convergence of the singular vectors using a technique motivated by alternating subspace iteration. We then derive a fixed point iteration for the evolution of the singular values and show linear convergence to the soft thresholded singular values of the original matrix. This last step requires a perturbation result for fixed point iterations which may be of independent interest.
- 8 August 2023
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Applied Dynamical Systems
- Vol. 22 (3) , 2082-2122
Learning Theory for Dynamical Systems
The task of modeling and forecasting a dynamical system is one of the oldest problems, and it remains challenging. Broadly, this task has two subtasks: extracting the full dynamical information from a partial observation, and then explicitly learning the dynamics from this information. We present a mathematical framework in which the dynamical information is represented in the form of an embedding. The framework combines the two subtasks using the language of spaces, maps, and commutations. The framework also unifies two of the most common learning paradigms: delay-coordinates and reservoir computing. We use this framework as a platform for two other investigations of the reconstructed system, its dynamical stability and the growth of error under iterations. We show that these questions are deeply tied to more fundamental properties of the underlying system, i.e., the behavior of matrix cocycles over the base dynamics, its nonuniform hyperbolic behavior, and its decay of correlations. Thus, our framework bridges the gap between universally observed behavior of dynamics modeling and the spectral, differential, and ergodic properties intrinsic to the dynamics.
- 8 August 2023
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Applied Dynamical Systems
- Vol. 22 (3) , 2082-2122
Learning Theory for Dynamical Systems
The task of modeling and forecasting a dynamical system is one of the oldest problems, and it remains challenging. Broadly, this task has two subtasks: extracting the full dynamical information from a partial observation, and then explicitly learning the dynamics from this information. We present a mathematical framework in which the dynamical information is represented in the form of an embedding. The framework combines the two subtasks using the language of spaces, maps, and commutations. The framework also unifies two of the most common learning paradigms: delay-coordinates and reservoir computing. We use this framework as a platform for two other investigations of the reconstructed system, its dynamical stability and the growth of error under iterations. We show that these questions are deeply tied to more fundamental properties of the underlying system, i.e., the behavior of matrix cocycles over the base dynamics, its nonuniform hyperbolic behavior, and its decay of correlations. Thus, our framework bridges the gap between universally observed behavior of dynamics modeling and the spectral, differential, and ergodic properties intrinsic to the dynamics.
- Open Access
- 1 August 2023
- Published by International Joint Conferences on Artificial Intelligence
- Vol. 4, 3651-3658
ActUp: Analyzing and Consolidating tSNE and UMAP
TSNE and UMAP are popular dimensionality reduction algorithms due to their speed and interpretable low-dimensional embeddings. Despite their popularity, however, little work has been done to study their full span of differences. We theoretically and experimentally evaluate the space of parameters in the TSNE and UMAP algorithms and observe that a single one -- the normalization -- is responsible for switching between them. This, in turn, implies that a majority of the algorithmic differences can be toggled without affecting the embeddings. We discuss the implications this has on several theoretic claims behind UMAP, as well as how to reconcile them with existing TSNE interpretations. Based on our analysis, we provide a method (GDR) that combines previously incompatible techniques from TSNE and UMAP and can replicate the results of either algorithm. This allows our method to incorporate further improvements, such as an acceleration that obtains either method's outputs faster than UMAP. We release improved versions of TSNE, UMAP, and GDR that are fully plug-and-play with the traditional libraries.
- 1 August 2023
- Published by International Joint Conferences on Artificial Intelligence
- Vol. 4, 3651-3658
Open AccessActUp: Analyzing and Consolidating tSNE and UMAP
TSNE and UMAP are popular dimensionality reduction algorithms due to their speed and interpretable low-dimensional embeddings. Despite their popularity, however, little work has been done to study their full span of differences. We theoretically and experimentally evaluate the space of parameters in the TSNE and UMAP algorithms and observe that a single one -- the normalization -- is responsible for switching between them. This, in turn, implies that a majority of the algorithmic differences can be toggled without affecting the embeddings. We discuss the implications this has on several theoretic claims behind UMAP, as well as how to reconcile them with existing TSNE interpretations. Based on our analysis, we provide a method (GDR) that combines previously incompatible techniques from TSNE and UMAP and can replicate the results of either algorithm. This allows our method to incorporate further improvements, such as an acceleration that obtains either method's outputs faster than UMAP. We release improved versions of TSNE, UMAP, and GDR that are fully plug-and-play with the traditional libraries.