Andreas Savas Tolias

Sensory neurons are traditionally viewed as feature detectors that respond with an increase in firing rate to preferred stimuli while remaining unresponsive to others. Here, we identify a dual-feature encoding strategy in macaque visual cortex, wherein many neurons in areas V1 and V4 are selectively tuned to two distinct visual features—one that enhances and one that suppresses activity—around an elevated baseline firing rate. By combining neuronal recordings with functional digital twin models—deep learning-based predictive models of biological neurons—we were able to systematically identify each neuron’s preferred and non-preferred features. These feature pairs served as anchors for a continuous, low-dimensional axis in natural image similarity space, along which neuronal activity varied approximately linearly. Within a single visual area, visual features that strongly or weakly activated individual neurons also had a high probability of modulating the activity of other neurons, suggesting a shared feature selectivity across the population that structures stimulus encoding. We show that this encoding strategy is conserved across species, present in both primary and lateral visual areas of mouse cortex. Dual-feature selectivity is consistent with recent anatomical evidence for feature-specific inhibitory connectivity, complementing the feature-detector principle through circuit mechanisms in which selective excitation and inhibition may together enhance the representational capacity of the neuronal population.

1 Concurrent advances in artificial intelligence (AI), large-scale neuronal recording, and behavioral tracking have enabled increasingly powerful predictive models of brain-behavior relationships. Yet predictive accuracy does not automatically yield interpretable principles of neural organization, particularly during natural behavior, where the relevant variables computed are often unknown in advance. Here, we developed a functional digital twin of the hippocampus, trained jointly on large-scale neural activity and self-directed three-dimensional flight in freely flying bats. A multi-modal, multitask transformer accurately predicted both neural activity and behavior, outperformed standard approaches, and generalized across recording sessions. Interrogating the model’s representations suggested a form of hippocampal generalization across self-selected trajectories that was not apparent in physical coordinates, but was plainly visible in the model’s low-dimensional latent space. We confirmed this organization directly in the recorded neural activity: neurons fired at corresponding stages of spatially distinct paths connecting the same origin and destination, preserving their relative firing order even as their firing fields shifted with the path. Analysis of neuron-specific readouts further identified functional relationships among neurons that generalized from flight to waking rest and sleep, brain states never used for training, and differed systematically between the hippocampal subregions CA1 and CA3. Together, these results show how AI-based functional digital twins can complement conventional analysis by generating testable hypotheses about neural organization during natural behavior.

In natural environments, animals actively sample visual information to guide behavior. Sensory feedback is dynamic and often requires active movements, whether saccading across the lines of this page or walking through a park. From high-acuity vision in hawks to low-acuity mice, many animals actively navigate to seek information, which can be called infotaxis. Although mice have relatively low-acuity vision, they still rely on sight for critical behaviors including navigation and prey capture. Yet, how sensitive they are to visual information and whether they perform infotaxis has not been established. Here, we develop a virtual reality object discrimination task to investigate visual decision-making under naturalistic conditions. We show that mice perform infotaxis by actively seeking out informative views to guide their choices. Stimulus manipulations confirm that this strategy is modulated by the amount of available visual information. These results reveal that mice use principled active strategies to resolve visual uncertainty, highlighting a key role for information-seeking in natural vision.

Sensory neurons are traditionally viewed as feature detectors that respond with an increase in firing rate to preferred stimuli while remaining unresponsive to others. Here, we identify a dual-feature encoding strategy in macaque visual cortex, wherein many neurons in areas V1 and V4 are selectively tuned to two distinct visual features—one that enhances and one that suppresses activity—around an elevated baseline firing rate. By combining neuronal recordings with functional digital twin models—deep learning-based predictive models of biological neurons—we were able to systematically identify each neuron’s preferred and non-preferred features. These feature pairs served as anchors for a continuous, low-dimensional axis in natural image similarity space, along which neuronal activity varied approximately linearly. Within a single visual area, visual features that strongly or weakly activated individual neurons also had a high probability of modulating the activity of other neurons, suggesting a shared feature selectivity across the population that structures stimulus encoding. We show that this encoding strategy is conserved across species, present in both primary and lateral visual areas of mouse cortex. Dual-feature selectivity is consistent with recent anatomical evidence for feature-specific inhibitory connectivity, complementing the feature-detector principle through circuit mechanisms in which selective excitation and inhibition may together enhance the representational capacity of the neuronal population.

Despite variations in appearance, we recognize objects robustly. Neuronal populations responding to objects presented under varying transformations form object manifolds, and hierarchically organized visual areas are hypothesized to untangle pixel intensities into linearly decodable object representations. However, the associated changes in the geometry of the object manifolds along the cortex remain unknown. Using home cage training, we showed that mice are capable of invariant object recognition. We recorded the responses of thousands of neurons to measure the information about object identity across the visual cortex and found that the lateral areas LM, LI, and AL carry more linearly decodable object information compared to other visual areas. We applied the theory of linear separability of manifolds and found that the increase in classification capacity is associated with a decrease in the dimension and radius of the object manifold, identifying the key features in the geometry of the population neural code that are associated with invariant object coding.

Sensory systems support generalization by representing features that persist under input variation; however, identifying the neuronal basis of these invariances remains difficult due to high-dimensional and nonlinear neural computations. Here we leverage the inception loop paradigm, iterating between large-scale recordings, predictive models and in silico experiments with in vivo verification, to characterize neuronal invariances in mouse primary visual cortex (V1). We synthesize varied exciting inputs (VEIs), dissimilar images that drive target neurons. These VEIs revealed a new bipartite invariance: one subfield encodes a shift-tolerant high-frequency texture and the other encodes a fixed low-frequency pattern. This division aligns with object boundaries defined by spatial frequency differences in highly activating images, suggesting a contribution to segmentation. Analysis of the MICrONS dataset revealed a hierarchy of excitatory neurons in mouse V1 layers 2/3: postsynaptic neurons exhibited greater invariance than their presynaptic inputs, while neurons with lower invariance formed more connections. Together, these results provide insights and scalable methodology for mapping neuronal invariances.

Deciphering the brain’s structure-function relationship is key to understanding the neuronal mechanisms underlying perception and cognition. The cortical column, a vertical organization of neurons with similar functions, is a classic example of primate neocortex structure-function organization. While columns have been identified in primary sensory areas using parametric stimuli, their prevalence across higher-level cortex is debated, particularly regarding complex tuning in natural image space. However, a key hurdle in identifying columns is characterizing the complex, nonlinear tuning of neurons to high-dimensional sensory inputs. Building on prior findings of topological organization for features like color and orientation, we investigate functional clustering in macaque visual area V4 in non-parametric natural image space, using large-scale recordings and deep learning–based analysis. We combined linear probe recordings with deep learning methods to systematically characterize the tuning of >1,200 V4 neurons using in silico synthesis of most exciting images (MEIs), followed by in vivo verification. Single V4 neurons exhibited MEIs containing complex features, including textures and shapes, and even high-level attributes with eye-like appearance. Neurons recorded on the same silicon probe, inserted orthogonal to the cortical surface, often exhibited similarities in their spatial feature selectivity, suggesting a degree of functional organization along the cortical depth. We quantified MEI similarity using human psychophysics and distances in a contrastive learning-derived embedding space. Moreover, the selectivity of the V4 neuronal population showed evidence of clustering into functional groups of shared feature selectivity. These functional groups showed parallels with the feature maps of units in artificial vision systems, suggesting potential shared encoding strategies. These results demonstrate the feasibility and scalability of deep learning–based functional characterization of neuronal selectivity in naturalistic visual contexts, offering a framework for quantitatively mapping cortical organization across multiple levels of the visual hierarchy.

Can general-purpose AI architectures go beyond prediction to discover the physical laws governing the universe? True intelligence relies on "world models" -- causal abstractions that allow an agent to not only predict future states but understand the underlying governing dynamics. While previous "AI Physicist" approaches have successfully recovered such laws, they typically rely on strong, domain-specific priors that effectively "bake in" the physics. Conversely, Vafa et al. recently showed that generic Transformers fail to acquire these world models, achieving high predictive accuracy without capturing the underlying physical laws. We bridge this gap by systematically introducing three minimal inductive biases. We show that ensuring spatial smoothness (by formulating prediction as continuous regression) and stability (by training with noisy contexts to mitigate error accumulation) enables generic Transformers to surpass prior failures and learn a coherent Keplerian world model, successfully fitting ellipses to planetary trajectories. However, true physical insight requires a third bias: temporal locality. By restricting the attention window to the immediate past -- imposing the simple assumption that future states depend only on the local state rather than a complex history -- we force the model to abandon curve-fitting and discover Newtonian force representations. Our results demonstrate that simple architectural choices determine whether an AI becomes a curve-fitter or a physicist, marking a critical step toward automated scientific discovery.