Tim Gollisch

Retinal ganglion cells, the output neurons of the vertebrate retina, often display nonlinear summation of visual signals over their receptive fields. This creates sensitivity to spatial contrast, letting the cells respond to spatially structured visual stimuli even when no net change in overall illumination of the receptive field occurs. Yet, computational models of ganglion cell responses are often based on linear receptive fields, and typical nonlinear extensions, which separate receptive fields into nonlinearly combined subunits, are often cumbersome to fit to experimental data. Previous work has suggested to model spatial-contrast sensitivity in responses to flashed images by combining signals from the mean and variance of light intensity inside the receptive field. Here, we extend and adjust this spatial contrast model for application to spatiotemporal stimulation and explore its performance on spiking responses that we recorded from ganglion cells of marmosets under artificial and naturalistic movies. We show how the model can be fitted to experimental data and that it outperforms common models with linear spatial integration to different degrees for different types of ganglion cells. Finally, we use the model framework to infer the cells’ spatial scale of nonlinear spatial integration. Our work shows that the spatial contrast model can capture aspects of nonlinear spatial integration in the primate retina with only few free parameters. The model can be used to assess the cells’ functional properties under natural stimulation and provides a simple-to-obtain benchmark for comparison with more detailed nonlinear encoding models.

The retina encodes a broad range of stimuli, adapting its computations to features like brightness, contrast, and motion. However, it is unclear whether it also adapts when switching between natural scenes and white noise (WN). To address this, we analyzed the neural activity of male marmoset retinal ganglion cells (RGCs) in response to WN and naturalistic movies. We trained linear–nonlinear models on both stimuli, evaluated their performance, and compared their receptive fields across stimulus domains. We found that models with spatial filters trained on one stimulus ensemble were less accurate when predicting neural activity on the other compared to models trained directly on the target stimulus. This suggests that spatial processing adapts to stimulus statistics. Different RGC types exhibited distinct changes: The OFF midget cells’ receptive fields became enlarged under natural movies (NMs), resulting in a lower cutoff frequency. Parasol cells and large OFF cells did not significantly change their receptive field sizes. All cell types exhibited stronger surrounds under NMs, resembling the whitening filters predicted by efficient coding for stimulus decorrelation, prompting us to test whether these changes were related to the different spectral content of the two stimulus types. Quantifying the effects of the filters’ enhanced surrounds on the stimulus power spectrum showed a significant contribution toward whitening only in ON parasol cells, where a whitening effect emerged regardless of the training stimulus. These results suggest that while RGCs adapt to the differences between WN and NM stimuli, efficient coding can only partially account for this adaptation.

Photoreceptor degeneration is among the leading causes of blindness and optogenetics as a potential therapeutic measure has garnered much attention over recent years. In this approach, light-sensitive molecules like Channelrhodopsin-2 (ChR2) are inserted into neurons in the retina to play the role of light-sensing elements after the loss of photoreceptors. Previous studies have shown that retinal ganglion cells (RGCs) in blind animal models with optogenetically modified retinas can respond reliably to steps in light intensity or similar diagnostic stimuli. Yet, little is known about how responses to natural stimuli in optogenetically treated retinas compare to normal, photoreceptor-mediated responses and how any differences might be counteracted by adjusting the stimulation. In this work, using mice of both sexes with intact photoreceptors as well as ChR2 expression in RGCs, we directly compared the encoding of natural images by individual RGCs under photoreceptor and optogenetic stimulation. We observed that evoked firing rates under optogenetic stimulation, relative to photoreceptor stimulation, often display reduced thresholding effects and a more linear dependence on receptive-field activation as well as reduced sensitivity to local spatial contrast and reduced dynamic range. Based on these differences, we devised modifications of the natural images, including thresholding and scaling of pixel intensities together with spatial low-pass filtering, and found that using such modified images under optogenetic stimulation can lead to stronger responses that are also more similar to the original photoreceptor-evoked responses. These findings may help optimize stimulation of optogenetically modified retinas to achieve more natural vision in future therapeutic applications. Significance Statement: Degenerative diseases that lead to the loss of photoreceptors, the eye’s light sensors in the retina, are a major cause of blindness. One promising therapy approach uses optogenetics to place light-sensitive proteins into retinal neurons, allowing them to detect light in the photoreceptors’ stead. We compared how the retina responds to natural images when stimulated via the inserted light-sensitive proteins versus normal activation and observed systematic differences in how images are represented, owing to reductions in response range, signal thresholding, and contrast sensitivity. Yet, by modifying the presented images, including spatial blurring as well as intensity thresholding and scaling, we managed to restore more natural image responses. These results suggest ways to improve visual quality from optogenetic treatments of blindness.

Zusammenfassung Hintergrund Für die Konzeption optogenetischer Therapieansätze bei degenerativen Erkrankungen der Netzhaut sind 2 Faktoren von großer Bedeutung: Erstens adressiert man eine kranke Netzhaut, deren Morphologie und Funktion durch die Krankheit bereits stark verändert sein kann. Dies bezeichnet man als Remodeling. Der zweite Faktor ist die komplexe Signalverarbeitung in der Netzhaut, die durch die Erkrankung selbst verändert oder durch die optogenetische Therapie kurzgeschlossen sein kann. Fragestellung In diesem Beitrag wird der aktuelle Wissensstand zu den krankheitsbedingten morphologischen und funktionellen Umbauprozessen der Netzhaut vorgestellt, und die Herausforderungen für optogenetische Therapieansätze werden diskutiert. Material und Methoden Es handelt sich um eine narrative Übersichtsarbeit. Ergebnisse Obwohl hereditäre Erkrankungen wie die Retinitis pigmentosa primär die äußere Netzhaut betreffen, erfassen die strukturellen und funktionellen Umbauprozesse im weiteren Verlauf die gesamte Netzhaut. Die Erkenntnisse hierzu stammen weitgehend aus tierexperimentellen Untersuchungen. Neueste, hochauflösende Bildgebungsmethoden könnten uns dabei helfen, das Remodeling auch beim Menschen besser zu verstehen. Dies wird es ermöglichen, bei Patienten mit fortgeschrittener Netzhautdegeneration stadienabhängig die optimale Zielzellpopulation für optogenetische Therapieansätze zu identifizieren. Schlussfolgerungen Umbauprozesse sowie auch die retinale Signalverarbeitung müssen bei der Konzeption optogenetischer Therapieansätze berücksichtigt werden. Perspektivisch ist die Entwicklung sowohl Ganglienzell- als auch Bipolarzell-gerichteter Ansätze sinnvoll, um je nach Krankheitsstadium die noch vorhandenen Funktionen optimal nutzen zu können.

Wide-field amacrine cells (ACs) play a unique role in retinal processing by integrating visual information across a large spatial area. Their inhibitory influence has been implicated in multiple retinal functions such as differential motion detection and the suppression of retinal activity during eye movements. However, a coherent understanding of their general function is lacking due to difficulties in directly recording from these cells and identifying effective visual stimuli to activate them. In this study, we used convolutional neural networks (CNNs) to investigate widefield inhibition mediated by wide-field ACs in the marmoset retina. We trained CNNs to mimic the function of the retina by predicting retinal ganglion cell (RGC) population responses to naturalistic movie stimuli and optimising the most exciting inputs (MEIs) to visualise RGCs’ receptive field (RF) structures. We then optimized suppressive surrounds beyond classical RGC RF boundaries, intended to capture the inhibitory effect of wide-field ACs on RGC activity. These optimized surrounds reduced MEI-elicited activity by 10% to 30%, demonstrating that CNNs not only mimic retinal responses but can also reveal hidden computational aspects of wide-field inhibition. However, suppression strength and generalization varied across architectures and datasets, indicating potential model-specific effects, high-lighting the importance of cautious interpretation. Overall, our approach illustrates how interpretability methods applied to artificial neural networks can offer new hypotheses regarding biological retinal computation, paving the way for targeted experimental validation. The code is available here .

The dichotomy of excitation and suppression is one of the canonical mechanisms explaining the complexity of neural activity. Computational models of the interplay of excitation and suppression in single neurons aim at investigating how this interaction affects a neuron’s spiking responses and shapes the encoding of sensory stimuli. Here, we compare the performance of three filter-based stimulus-encoding models for predicting retinal ganglion cell responses recorded from axolotl, mouse, and marmoset retina to different types of temporally varying visual stimuli. Suppression in these models is implemented via subtractive or divisive interactions of stimulus filters or by a response-driven feedback module. For the majority of ganglion cells, the subtractive and divisive models perform similarly and outperform the feedback model as well as a linear-nonlinear (LN) model with no suppression. Comparison between the subtractive and the divisive model depends on cell type, species, and stimulus components, with the divisive model generalizing best across temporal stimulus frequencies and visual contrast and the subtractive model capturing in particular responses for slow temporal stimulus dynamics and for slow axolotl cells. Overall, we conclude that the divisive and subtractive models are well suited for capturing interactions of excitation and suppression in ganglion cells and perform best for different temporal regimes of these interactions.

The retina encodes a broad range of stimuli, adapting its computations to features like brightness, contrast, or motion. However, it is unclear to what extent it also adapts to spatial frequency content – as theories of efficient coding would predict – for instance, when switching between natural scenes and white noise. To address this, we analyzed neural activity of marmoset retinal ganglion cells (RGCs) in response to white noise and naturalistic movie stimuli. We trained linear-nonlinear models on both stimuli, evaluated their performance and compared their receptive fields (RFs) across the stimulus domains. We found that the models with spatial filters trained on either one of the stimulus ensembles were not able to predict the neural activity on the other as accurately as the models trained on the target stimulus. This suggests that spatial processing adapts to stimulus statistics. Different RGC types exhibited distinct changes: the midget OFF cells’ RFs became enlarged under natural movie statistics, resulting in a lower cutoff frequency. Parasol cells did not change their RF size significantly. Large OFF cells’ RFs decreased in size. All cell types exhibited stronger surrounds under natural movies, resembling the whitening filters predicted by efficient coding. However, quantifying the effect of the filter adaptation on the stimulus power spectrum showed a significant contribution towards whitening only in ON parasol cells. The whitening effect emerged regardless of the training stimulus. These results suggest that while RGCs adapt to the spatial frequency content of the input, efficient coding can only partially account for this adaptation. Significance statement Natural scenes differ from artificial stimuli like white noise, in spatial frequency structure. How the retina adapts to these differences remains unclear. To explore this, we studied responses of four primate retinal ganglion cell types to artificial and natural stimuli. Our results show that some cell types, like midget cells, enlarge their receptive fields and surrounds under natural stimuli, while others, like parasol cells, only enhance surrounds. These changes align qualitatively with the efficient coding theory, which posits redundancy reduction. However, in three cell types, the enhanced surrounds did not significantly whiten responses to natural stimuli, contrary to efficient coding predictions. These findings challenge how fully efficient coding explains retinal adaptation, suggesting that other principles underlie the processing of visual inputs.

The role of the vertebrate retina in early vision is generally described by the efficient coding hypothesis^1,2, which predicts that the retina reduces the redundancy inherent in natural scenes³ by discarding spatiotemporal correlations while preserving stimulus information⁴. It is unclear, however, whether the predicted decorrelation and redundancy reduction in the activity of ganglion cells, the retina’s output neurons, hold under gaze shifts, which dominate the dynamics of the natural visual input⁵. We show here that species-specific gaze patterns in natural stimuli can drive correlated spiking responses both in and across distinct types of ganglion cells in marmoset as well as mouse retina. These concerted responses disrupt redundancy reduction to signal fixation periods with locally high spatial contrast. Model-based analyses of ganglion cell responses to natural stimuli show that the observed response correlations follow from nonlinear pooling of ganglion cell inputs. Our results indicate cell-type-specific deviations from efficient coding in retinal processing of natural gaze shifts.

Spatially nonlinear stimulus integration by retinal ganglion cells lies at the heart of various computations performed by the retina. It arises from the nonlinear transmission of signals that ganglion cells receive from bipolar cells, which thereby constitute functional subunits within a ganglion cell’s receptive field. Inferring these subunits from recorded ganglion cell activity promises a new avenue for studying the functional architecture of the retina. This calls for efficient methods, which leave sufficient experimental time to leverage the acquired knowledge for further investigating identified subunits. Here, we combine concepts from super-resolution microscopy and computed tomography and introduce super-resolved tomographic reconstruction (STR) as a technique to efficiently stimulate and locate receptive field subunits. Simulations demonstrate that this approach can reliably identify subunits across a wide range of model variations, and application in recordings of primate parasol ganglion cells validates the experimental feasibility. STR can potentially reveal comprehensive subunit layouts within only a few tens of minutes of recording time, making it ideal for online analysis and closed-loop investigations of receptive field substructure in retina recordings.

A standard circuit motif in sensory systems is the pooling of sensory information from an upstream neuronal layer. A downstream neuron thereby collects signals across different locations in stimulus space, which together compose the neuron’s receptive field. In addition, nonlinear transformations in the signal transfer between the layers give rise to functional subunits inside the receptive field. For ganglion cells in the vertebrate retina, for example, receptive field subunits are thought to correspond to presynaptic bipolar cells. Identifying the number and locations of subunits from the stimulus–response relationship of a recorded ganglion cell has been an ongoing challenge in order to characterize the retina’s functional circuitry and to build computational models that capture nonlinear signal pooling. Here we present a novel version of spike-triggered non-negative matrix factorization (STNMF), which can extract localized subunits in ganglion-cell receptive fields from recorded spiking responses under spatiotemporal white-noise stimulation. The method provides a more than 100-fold speed increase compared to a previous implementation, which can be harnessed for systematic screening of hyperparameters, such as sparsity regularization. We demonstrate the power and flexibility of this approach by analyzing populations of ganglion cells from salamander and primate retina. We find that subunits of midget as well as parasol ganglion cells in the marmoset retina form separate mosaics that tile visual space. Moreover, subunit mosaics show alignment with each other for ON and OFF midget as well as for ON and OFF parasol cells, indicating a spatial coordination of ON and OFF signals at the bipolar-cell level. Thus, STNMF can reveal organizational principles of signal transmission between successive neural layers, which are not easily accessible by other means.