Christoph Schnörr

This paper introduces the sigma flow model for the prediction of structured labelings of data observed on Riemannian manifolds, including Euclidean image domains as special case. The approach combines the Laplace–Beltrami framework for image denoising and enhancement, introduced by Sochen, Kimmel and Malladi about 25 years ago, and the assignment flow approach introduced and studied by the authors. The sigma flow arises as the Riemannian gradient flow of generalized harmonic energies and is thus governed by a nonlinear geometric PDE which determines a harmonic map from a closed Riemannian domain manifold to a statistical manifold, equipped with the Fisher–Rao metric from information geometry. A specific ingredient of the sigma flow is the mutual dependency of the Riemannian metric of the domain manifold on the evolving state. This makes the approach amenable to machine learning in a specific way, by realizing this dependency through a mapping with compact time-variant parametrization that can be learned from data. Proof-of-concept experiments demonstrate the expressivity of the sigma flow model and prediction performance. Structural similarities to transformer network architectures and networks generated by the geometric integration of sigma flows are pointed out, which highlights the connection to deep learning and, conversely, may stimulate the use of geometric design principles for structured prediction in other areas of scientific machine learning.

We introduce a novel generative model for the representation of joint probability distributions of a possibly large number of discrete random variables. The approach uses measure transport by randomized assignment flows on the statistical submanifold of factorizing distributions, which enables to represent and sample efficiently from any target distribution and to assess the likelihood of unseen data points. The complexity of the target distribution only depends on the parametrization of the affinity function of the dynamical assignment flow system. Our model can be trained in a simulation-free manner by conditional Riemannian flow matching, using the training data encoded as geodesics on the assignment manifold in a closed form, with respect to the e-connection of information geometry. Numerical experiments devoted to distributions of structured image labelings demonstrate the applicability to large-scale problems, which may include discrete distributions in other application areas. Performance measures show that our approach scales better with the increasing number of classes than the recent related work.

Moving object detection and segmentation from a single moving camera is a challenging task, requiring an understanding of recognition, motion and 3D geometry. Combining both recognition and reconstruction boils down to a fusion problem, where appearance and motion features need to be combined for classification and segmentation. In this paper, we present a novel fusion architecture for monocular motion segmentation - M3Former, which leverages the strong performance of transformers for segmentation and multi-modal fusion. As reconstructing motion from monocular video is ill-posed, we systematically analyze different 2D and 3D motion representations for this problem and their importance for segmentation performance. Finally, we analyze the effect of training data and show that diverse datasets are required to achieve SotA performance on Kitti and Davis.

Physical systems can be used as an information processing substrate and, with that, extend traditional computing architectures. For such an application, the experimental platform must guarantee pristine control of the initial state, the temporal evolution, and readout. All these ingredients are provided by modern experimental realizations of atomic Bose-Einstein condensates. By embedding a quantum gas experiment in a machine learning pipeline, one can represent nonlinear functions while only linear operations on classical computers of the pipeline are necessary. We demonstrate successful regression and interpolation of a nonlinear function using an elongated cloud of potassium atoms and characterize the performance of our system. Published by the American Physical Society 2024

Metric data labeling refers to the task of assigning one of multiple predefined labels to every given datapoint based on the metric distance between label and data. This assignment of labels typically takes place in a spatial or spatio-temporal context. Assignment flows are a class of dynamical models for metric data labeling that evolve on a basic statistical manifold, the so called assignment manifold, governed by a system of coupled replicator equations. In this paper we generalize the result of a recent paper for uncoupled replicator equations and adopting the viewpoint of geometric mechanics, relate assignment flows to critical points of an action functional via the associated Euler–Lagrange equation. We also show that not every assignment flow is a critical point and characterize precisely the class of coupled replicator equations fulfilling this relation, a condition that has been missing in recent related work. Finally, some consequences of this connection to Lagrangian mechanics are investigated including the fact that assignment flows are, up to initial conditions of measure zero, reparametrized geodesics of the so-called Jacobi metric.

This paper introduces assignment flows for density matrices as state spaces for representation and analysis of data associated with vertices of an underlying weighted graph. Determining an assignment flow by geometric integration of the defining dynamical system causes an interaction of the non-commuting states across the graph, and the assignment of a pure (rank-one) state to each vertex after convergence. Adopting the Riemannian–Bogoliubov–Kubo–Mori metric from information geometry leads to closed-form local expressions that can be computed efficiently and implemented in a fine-grained parallel manner. Restriction to the submanifold of commuting density matrices recovers the assignment flows for categorical probability distributions, which merely assign labels from a finite set to each data point. As shown for these flows in our prior work, the novel class of quantum state assignment flows can also be characterized as Riemannian gradient flows with respect to a non-local, non-convex potential after proper reparameterization and under mild conditions on the underlying weight function. This weight function generates the parameters of the layers of a neural network corresponding to and generated by each step of the geometric integration scheme. Numerical results indicate and illustrate the potential of the novel approach for data representation and analysis, including the representation of correlations of data across the graph by entanglement and tensorization.

The Turing mechanism describes the emergence of spatial patterns due to spontaneous symmetry breaking in reaction–diffusion processes and underlies many developmental processes. Identifying Turing mechanisms in biological systems defines a challenging problem. This paper introduces an approach to the prediction of Turing parameter values from observed Turing patterns. The parameter values correspond to a parametrized system of reaction–diffusion equations that generate Turing patterns as steady state. The Gierer–Meinhardt model with four parameters is chosen as a case study. A novel invariant pattern representation based on resistance distance histograms is employed, along with Wasserstein kernels, in order to cope with the highly variable arrangement of local pattern structure that depends on the initial conditions which are assumed to be unknown. This enables us to compute physically plausible distances between patterns, to compute clusters of patterns and, above all, model parameter prediction based on training data that can be generated by numerical model evaluation with random initial data: for small training sets, classical state-of-the-art methods including operator-valued kernels outperform neural networks that are applied to raw pattern data, whereas for large training sets the latter are more accurate. A prominent property of our approach is that only a single pattern is required as input data for model parameter predicion. Excellent predictions are obtained for single parameter values and reasonably accurate results for jointly predicting all four parameter values.