Pratik Chaudhari

This paper develops a mathematical argument and algorithms for building representations of data from event-based cameras that we call Fast Feature Field (F ³ ). We learned this representation by predicting future events from past events and show that it preserves scene structure and motion information. F ³ exploits the sparsity of event data and is robust to noise and variations in event rates. It can be computed efficiently using ideas from multiresolution hash encoding and deep sets, achieving 120 hertz at high-definition (HD) resolution and 440 hertz at video graphics array (VGA) resolution. F ³ represents events in a contiguous spatiotemporal volume as a multichannel image, enabling a range of downstream tasks. We obtained state-of-the-art performance on optical flow estimation, semantic segmentation, and monocular metric depth estimation, on data from three robotic platforms (a car, a quadruped robot, and a flying platform) and across different lighting conditions (daytime and nighttime), environments (indoors, outdoors, urban, and off-road), and dynamic vision sensors (resolutions and event rates). Our implementations can predict these tasks at 25 to 75 hertz at HD resolution.

Neural dynamical systems (NDSs) are expressive temporal predictors that capture continuous-time dynamics through fine-grained state updates. However, this sequential structure maps poorly onto digital hardware optimized for dense matrix operations, a mismatch that analog neuromorphic computing, with its native continuous-time dynamics, can resolve. We introduce FerroNDS, a neuromorphic system built from two analog primitives: an integrator for temporal accumulation and an oscillator for frequency-selective filtering. We map this system onto compute-in-memory (CIM) hardware based on multi-bit ferrodiodes. A 48-unit FerroNDS bank computes a short-time Fourier transform whose spectrogram correlates at 0.93 with a Hann reference, while a 32-unit instance forecasts frequency and fault power at horizons from 14.5 to 228.5 ms and chaotic dynamics over a rollout of 10 Lyapunov times. Both results use behavioral models of the analog primitives, fitted and validated against LTspice traces of the full circuit. The system achieves sub-watt, real-time operation with per-neuron per-inference energy of 82.9–89.5 nJ (200 Hz) and 14.7–14.8 nJ (10 kHz), with per-layer latency of 3.18 ms (200 Hz) and 63.66 μ s (10 kHz). To our knowledge, this is the first end-to-end integration of a ferrodiode into a neuromorphic computational framework, establishing ferroelectric CIM as a practical substrate for analog NDSs.

Image matching is a fundamental task in quantitative biomedical and biological image analyses, enabling researchers to compare, integrate, and interpret imaging data across subjects, time points, modalities, and experimental conditions. Existing state-of-the-art registration methods are slow due to inefficient implementations and poor convergence rates because of the ill-conditioned nature of the optimization problem. Deep learning methods offer fast inference but require extensive training time, substantial inference memory, and fail to generalize across long-tailed distributions or diverse image modalities, necessitating costly retraining. We address these challenges by proposing FireANTs, a training-free, GPU-accelerated, multi-scale adaptive Riemannian optimization algorithm for fast and accurate dense diffeomorphic image matching. FireANTs more than doubles the speed of the community standard ANTs registration tool on a CPU, and is two orders of magnitude faster on a GPU. On the GPU, FireANTs performs competitively with deep learning methods on inference runtime while consuming up to 10 × less memory. FireANTs demonstrates robustness on a wide variety of matching problems across modalities, species, and organs, without any domain-specific training or tuning. Our framework allows hyperparameter grid search studies with less resources and time compared to traditional and deep learning registration algorithms alike.

The challenge of scaling digital computing motivates innovation, especially through the evolution of physical systems that mimic neural networks and combinatorial optimization problems. Light is a hyperefficient information carrier, and if efficient interactions with it could be uncovered, then direct information processing would become far more feasible. We harness an ensemble of hundreds of Kerr microresonator solitons and implement an analog feedback network to create an Ising machine with fully programmable all-to-all interactions. By increasing the feedback for self, on-diagonal interactions, each soliton exhibits a universal spin-like bifurcation, and using this palette of interactions, we solve the canonical Boolean satisfiability problem (SAT). The combination of uniform soliton interactions and the compatibility of our Ising machine with high-speed data interconnects enables rapid and precise solutions of complex SAT problems. The well-established theoretical properties of Kerr solitons bound the trade-off of optical power and time use by the machine at ~0.15 milliwatts per soliton and 1 microsecond for a single feedback step. We performed >10,000 trials on more than 100 randomly generated SAT instances to evaluate the Ising machine, demonstrating the potential to exceed the performance of benchmark digital SAT solvers. Our work highlights the convergence of optical nonlinearity, ultralow loss photonics, and optoelectronic circuits for computation-acceleration tasks.

Neurosim is a fast, real-time, high-performance library for simulating sensors such as dynamic vision sensors, RGB cameras, depth sensors, and inertial sensors. It can also simulate agile dynamics of multi-rotor vehicles in complex and dynamic environments. Neurosim can achieve frame rates as high as ~2700 FPS on a desktop GPU. Neurosim integrates with a ZeroMQ-based communication library called Cortex to facilitate seamless integration with machine learning and robotics workflows. Cortex provides a high-throughput, low-latency message-passing system for Python and C++ applications, with native support for NumPy arrays and PyTorch tensors. This paper discusses the design philosophy behind Neurosim and Cortex. It demonstrates how they can be used to (i) train neuromorphic perception and control algorithms, e.g., using self-supervised learning on time-synchronized multi-modal data, and (ii) test real-time implementations of these algorithms in closed-loop. Neurosim and Cortex are available at https://github.com/grasp-lyrl/neurosim .

Recent experiments have shown that training trajectories of multiple deep neural networks with different architectures, optimization algorithms, hyperparameter settings, and regularization methods evolve on a remarkably low-dimensional "hyperribbon-like" manifold in the space of probability distributions. Inspired by the similarities in the training trajectories of deep networks and linear networks, we analytically characterize this phenomenon for the latter. We show, using tools in dynamical systems theory, that the geometry of this low-dimensional manifold is controlled by (i) the decay rate of the eigenvalues of the input correlation matrix of the training data, (ii) the relative scale of the ground-truth output to the weights at the beginning of training, and (iii) the number of steps of gradient descent. By analytically computing and bounding the contributions of these quantities, we characterize phase boundaries of the region where hyperribbons are to be expected. We also extend our analysis to kernel machines and linear models that are trained with stochastic gradient descent.

The LUMIR challenge represents an important benchmark for evaluating deformable image registration methods on large-scale neuroimaging data. While the challenge demonstrates that modern deep learning methods achieve competitive accuracy on T1-weighted MRI, it also claims exceptional zero-shot generalization to unseen contrasts and resolutions, assertions that contradict established understanding of domain shift in deep learning. In this paper, we perform an independent re-evaluation of these zero-shot claims using rigorous evaluation protocols while addressing potential sources of instrumentation bias. Our findings reveal a more nuanced picture: (1) deep learning methods perform comparably to iterative optimization on in-distribution T1w images and even on human-adjacent species (macaque), demonstrating improved task understanding; (2) however, performance degrades significantly on out-of-distribution contrasts (T2, T2*, FLAIR), with Cohen's d scores ranging from 0.7-1.5, indicating substantial practical impact on downstream clinical workflows; (3) deep learning methods face scalability limitations on high-resolution data, failing to run on 0.6 mm isotropic images, while iterative methods benefit from increased resolution; and (4) deep methods exhibit high sensitivity to preprocessing choices. These results align with the well-established literature on domain shift and suggest that claims of universal zero-shot superiority require careful scrutiny. We advocate for evaluation protocols that reflect practical clinical and research workflows rather than conditions that may inadvertently favor particular method classes.

Occam’s razor is the principle that, all else being equal, simpler explanations should be preferred over more complex ones. This principle is thought to guide human decision-making, but the nature of this guidance is not known. Here we used preregistered behavioral experiments to show that people tend to prefer the simpler of two alternative explanations for uncertain data. These preferences match predictions of formal theories of model selection that penalize excessive flexibility. These penalties emerge when considering not just the best explanation but the integral over all possible, relevant explanations. We further show that these simplicity preferences persist in humans, but not in certain artificial neural networks, even when they are maladaptive. Our results imply that principled notions of statistical model selection, including integrating over possible, latent causes to avoid overfitting to noisy observations, may play a central role in human decision-making.