Will Xiao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Will Xiao has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Gabriel Kreiman
Boston Children’s Hospital, Harvard Medical School, Boston, USA
8 shared publications - Margaret S. Livingstone
Harvard Medical School
7 shared publications - Carlos R. Ponce
Department of Neurobiology, Harvard Medical School
5 shared publications Giorgia Dellaferrera
Children’s Hospital, Harvard Medical School
3 shared publications- Dianna Hidalgo
Department of Neurobiology, Harvard Medical School
2 shared publications Gabriel Kreiman
Boston Children's Hospital Harvard Medical School 02115 Boston MA U.S.A
2 shared publications
Latest Publications
- Open Access
- 17 April 2026
- Published by eLife Sciences Publications, Ltd in eLife preprints
- Vol. 14
Trial-by-trial inter-areal interactions in visual cortex in the presence or absence of visual stimulation
State-of-the-art computational models of vision largely focus on fitting trial-averaged spike counts to visual stimuli using overparameterized neural networks. However, a computational model of the visual cortex should predict the dynamic responses of neurons in single trials across different experimental conditions. In this study, we investigated trial-by-trial inter-areal interactions in the visual cortex by predicting neuronal activity in one area based on activity in another, distinguishing between stimulus-driven and non-stimulus-driven shared variability. We analyzed two datasets: calcium imaging from mouse V1 layers 2/3 and 4, and extracellular neurophysiological recordings from macaque V1 and V4. Our results show that neuronal activity can be predicted bidirectionally between L2/3 and L4 in mice, and between V1 and V4 in macaque monkeys, with the latter interaction exhibiting directional asymmetry. The predictability of neuronal responses varied with the type of visual stimulus, yet responses could also be predicted in the absence of visual stimulation. In mice, we observed a bimodal distribution of neurons, with some neurons primarily driven by visual inputs and others showing predictable activity during spontaneous activity despite lacking consistent visually evoked responses. Predictability also depended on intrinsic neuronal properties, receptive field overlap, and the relative timing of activity across areas. Our findings highlight the presence of both stimulus- and non-stimulus-related components in interactions between visual areas across diverse contexts and underscore the importance of non-visual shared variability between visual regions in both mice and macaques.
- 17 April 2026
- Published by eLife Sciences Publications, Ltd in eLife preprints
- Vol. 14
Open AccessTrial-by-trial inter-areal interactions in visual cortex in the presence or absence of visual stimulation
State-of-the-art computational models of vision largely focus on fitting trial-averaged spike counts to visual stimuli using overparameterized neural networks. However, a computational model of the visual cortex should predict the dynamic responses of neurons in single trials across different experimental conditions. In this study, we investigated trial-by-trial inter-areal interactions in the visual cortex by predicting neuronal activity in one area based on activity in another, distinguishing between stimulus-driven and non-stimulus-driven shared variability. We analyzed two datasets: calcium imaging from mouse V1 layers 2/3 and 4, and extracellular neurophysiological recordings from macaque V1 and V4. Our results show that neuronal activity can be predicted bidirectionally between L2/3 and L4 in mice, and between V1 and V4 in macaque monkeys, with the latter interaction exhibiting directional asymmetry. The predictability of neuronal responses varied with the type of visual stimulus, yet responses could also be predicted in the absence of visual stimulation. In mice, we observed a bimodal distribution of neurons, with some neurons primarily driven by visual inputs and others showing predictable activity during spontaneous activity despite lacking consistent visually evoked responses. Predictability also depended on intrinsic neuronal properties, receptive field overlap, and the relative timing of activity across areas. Our findings highlight the presence of both stimulus- and non-stimulus-related components in interactions between visual areas across diverse contexts and underscore the importance of non-visual shared variability between visual regions in both mice and macaques.
- Open Access
Response dynamics in macaque ventral stream recapitulate the visual hierarchy
Chronic recording in macaque inferotemporal cortex (IT) revealed a progression of feature selectivity over the duration of the response, with different images evoking strongest responses in different time bins. Neurons responded briskly and transiently to early best images and gave slower and more sustained responses to later best images. Correlations between IT selectivity and deep neural network (DNN) activations in early layers were faster and more transient than correlations with deeper DNN layers, suggesting a hierarchical representation unfolding. Analysis of best images in different time bins showed that early best images were simpler, or made up of fewer parts, than later best images. This simple-to-complex selectivity dynamic is consistent with, but more universal than, previously reported dynamics of global-to-local or category-to-exemplar selectivity. In parallel with the simple-to-complex selectivity progression, the image selectivity and the population activity became sparser during the response, consistent with the hypothesis of a canonical cortical microcircuit that pools inputs and incorporates neighboring-neuron inhibition. We propose that this temporal cascade in IT recapitulates the multiple antecedent stages in the visual hierarchy, which also show a simple-to-complex progression of feature selectivity.
Open AccessResponse dynamics in macaque ventral stream recapitulate the visual hierarchy
Chronic recording in macaque inferotemporal cortex (IT) revealed a progression of feature selectivity over the duration of the response, with different images evoking strongest responses in different time bins. Neurons responded briskly and transiently to early best images and gave slower and more sustained responses to later best images. Correlations between IT selectivity and deep neural network (DNN) activations in early layers were faster and more transient than correlations with deeper DNN layers, suggesting a hierarchical representation unfolding. Analysis of best images in different time bins showed that early best images were simpler, or made up of fewer parts, than later best images. This simple-to-complex selectivity dynamic is consistent with, but more universal than, previously reported dynamics of global-to-local or category-to-exemplar selectivity. In parallel with the simple-to-complex selectivity progression, the image selectivity and the population activity became sparser during the response, consistent with the hypothesis of a canonical cortical microcircuit that pools inputs and incorporates neighboring-neuron inhibition. We propose that this temporal cascade in IT recapitulates the multiple antecedent stages in the visual hierarchy, which also show a simple-to-complex progression of feature selectivity.
- Open Access
Trial-by-trial inter-areal interactions in visual cortex in the presence or absence of visual stimulation
State-of-the-art computational models of vision largely focus on fitting trial-averaged spike counts to visual stimuli using overparameterized neural networks. However, a computational model of the visual cortex should predict the dynamic responses of neurons in single trials across different experimental conditions. In this study, we investigated trial-by-trial inter-areal interactions in the visual cortex by predicting neuronal activity in one area based on activity in another, distinguishing between stimulus-driven and non-stimulus-driven shared variability. We analyzed two datasets: calcium imaging from mouse V1 layers 2/3 and 4, and extracellular neurophysiological recordings from macaque V1 and V4. Our results show that neuronal activity can be predicted bidirectionally between L2/3 and L4 in mice, and between V1 and V4 in macaque monkeys, with the latter interaction exhibiting directional asymmetry. The predictability of neuronal responses varied with the type of visual stimulus, yet responses could also be predicted in the absence of visual stimulation. In mice, we observed a bimodal distribution of neurons, with some neurons primarily driven by visual inputs and others showing predictable activity during spontaneous activity despite lacking consistent visually evoked responses. Predictability also depended on intrinsic neuronal properties, receptive field overlap, and the relative timing of activity across areas. Our findings highlight the presence of both stimulus- and non-stimulus-related components in interactions between visual areas across diverse contexts and underscore the importance of non-visual shared variability between visual regions in both mice and macaques.
Open AccessTrial-by-trial inter-areal interactions in visual cortex in the presence or absence of visual stimulation
State-of-the-art computational models of vision largely focus on fitting trial-averaged spike counts to visual stimuli using overparameterized neural networks. However, a computational model of the visual cortex should predict the dynamic responses of neurons in single trials across different experimental conditions. In this study, we investigated trial-by-trial inter-areal interactions in the visual cortex by predicting neuronal activity in one area based on activity in another, distinguishing between stimulus-driven and non-stimulus-driven shared variability. We analyzed two datasets: calcium imaging from mouse V1 layers 2/3 and 4, and extracellular neurophysiological recordings from macaque V1 and V4. Our results show that neuronal activity can be predicted bidirectionally between L2/3 and L4 in mice, and between V1 and V4 in macaque monkeys, with the latter interaction exhibiting directional asymmetry. The predictability of neuronal responses varied with the type of visual stimulus, yet responses could also be predicted in the absence of visual stimulation. In mice, we observed a bimodal distribution of neurons, with some neurons primarily driven by visual inputs and others showing predictable activity during spontaneous activity despite lacking consistent visually evoked responses. Predictability also depended on intrinsic neuronal properties, receptive field overlap, and the relative timing of activity across areas. Our findings highlight the presence of both stimulus- and non-stimulus-related components in interactions between visual areas across diverse contexts and underscore the importance of non-visual shared variability between visual regions in both mice and macaques.
- Open Access
- 29 April 2024
- Published by Springer Nature in Nature Neuroscience
- Vol. 27 (6) , 1157-1166
Feature-selective responses in macaque visual cortex follow eye movements during natural vision
In natural vision, primates actively move their eyes several times per second via saccades. It remains unclear whether, during this active looking, visual neurons exhibit classical retinotopic properties, anticipate gaze shifts or mirror the stable quality of perception, especially in complex natural scenes. Here, we let 13 monkeys freely view thousands of natural images across 4.6 million fixations, recorded 883 h of neuronal responses in six areas spanning primary visual to anterior inferior temporal cortex and analyzed spatial, temporal and featural selectivity in these responses. Face neurons tracked their receptive field contents, indicated by category-selective responses. Self-consistency analysis showed that general feature-selective responses also followed eye movements and remained gaze-dependent over seconds of viewing the same image. Computational models of feature-selective responses located retinotopic receptive fields during free viewing. We found limited evidence for feature-selective predictive remapping and no viewing-history integration. Thus, ventral visual neurons represent the world in a predominantly eye-centered reference frame during natural vision.
- 29 April 2024
- Published by Springer Nature in Nature Neuroscience
- Vol. 27 (6) , 1157-1166
Open AccessFeature-selective responses in macaque visual cortex follow eye movements during natural vision
In natural vision, primates actively move their eyes several times per second via saccades. It remains unclear whether, during this active looking, visual neurons exhibit classical retinotopic properties, anticipate gaze shifts or mirror the stable quality of perception, especially in complex natural scenes. Here, we let 13 monkeys freely view thousands of natural images across 4.6 million fixations, recorded 883 h of neuronal responses in six areas spanning primary visual to anterior inferior temporal cortex and analyzed spatial, temporal and featural selectivity in these responses. Face neurons tracked their receptive field contents, indicated by category-selective responses. Self-consistency analysis showed that general feature-selective responses also followed eye movements and remained gaze-dependent over seconds of viewing the same image. Computational models of feature-selective responses located retinotopic receptive fields during free viewing. We found limited evidence for feature-selective predictive remapping and no viewing-history integration. Thus, ventral visual neurons represent the world in a predominantly eye-centered reference frame during natural vision.
- Open Access
Out of sight, out of mind: Responses in primate ventral visual cortex track individual fixations during natural vision
During natural vision, primates shift their gaze several times per second with large, ballistic eye movements known as saccades. Open questions remain as to whether visual neurons retain their classical retinotopic response properties during natural vision or whether neurons integrate information across fixations and predict the consequences of impending saccades. Answers are especially wanting for vision in complex scenes relevant to natural behavior. We let 13 monkeys freely view thousands of large natural images, recorded over 883 hours of neuronal responses throughout the ventral visual pathway across 4.7 million fixations, and designed flexible analyses to reveal the spatial, temporal, and feature selectivity of the responses. Ventral visual responses followed each fixation and did not become gaze-invariant as monkeys examined an image over seconds. Computational models revealed that neuronal responses corresponded to eye-centered receptive fields. The results suggest that ventral visual cortex remains predominantly retinotopic during natural vision and does not establish a gaze-independent representation of the world.
Open AccessOut of sight, out of mind: Responses in primate ventral visual cortex track individual fixations during natural vision
During natural vision, primates shift their gaze several times per second with large, ballistic eye movements known as saccades. Open questions remain as to whether visual neurons retain their classical retinotopic response properties during natural vision or whether neurons integrate information across fixations and predict the consequences of impending saccades. Answers are especially wanting for vision in complex scenes relevant to natural behavior. We let 13 monkeys freely view thousands of large natural images, recorded over 883 hours of neuronal responses throughout the ventral visual pathway across 4.7 million fixations, and designed flexible analyses to reveal the spatial, temporal, and feature selectivity of the responses. Ventral visual responses followed each fixation and did not become gaze-invariant as monkeys examined an image over seconds. Computational models revealed that neuronal responses corresponded to eye-centered receptive fields. The results suggest that ventral visual cortex remains predominantly retinotopic during natural vision and does not establish a gaze-independent representation of the world.
- Open Access
- 22 November 2022
- Published by Public Library of Science (PLoS) in PLOS Computational Biology
- Vol. 18 (11) , e1010654
Look twice: A generalist computational model predicts return fixations across tasks and species
Primates constantly explore their surroundings via saccadic eye movements that bring different parts of an image into high resolution. In addition to exploring new regions in the visual field, primates also make frequent return fixations, revisiting previously foveated locations. We systematically studied a total of 44,328 return fixations out of 217,440 fixations. Return fixations were ubiquitous across different behavioral tasks, in monkeys and humans, both when subjects viewed static images and when subjects performed natural behaviors. Return fixations locations were consistent across subjects, tended to occur within short temporal offsets, and typically followed a 180-degree turn in saccadic direction. To understand the origin of return fixations, we propose a proof-of-principle, biologically-inspired and image-computable neural network model. The model combines five key modules: an image feature extractor, bottom-up saliency cues, task-relevant visual features, finite inhibition-of-return, and saccade size constraints. Even though there are no free parameters that are fine-tuned for each specific task, species, or condition, the model produces fixation sequences resembling the universal properties of return fixations. These results provide initial steps towards a mechanistic understanding of the trade-off between rapid foveal recognition and the need to scrutinize previous fixation locations.
- 22 November 2022
- Published by Public Library of Science (PLoS) in PLOS Computational Biology
- Vol. 18 (11) , e1010654
Open AccessLook twice: A generalist computational model predicts return fixations across tasks and species
Primates constantly explore their surroundings via saccadic eye movements that bring different parts of an image into high resolution. In addition to exploring new regions in the visual field, primates also make frequent return fixations, revisiting previously foveated locations. We systematically studied a total of 44,328 return fixations out of 217,440 fixations. Return fixations were ubiquitous across different behavioral tasks, in monkeys and humans, both when subjects viewed static images and when subjects performed natural behaviors. Return fixations locations were consistent across subjects, tended to occur within short temporal offsets, and typically followed a 180-degree turn in saccadic direction. To understand the origin of return fixations, we propose a proof-of-principle, biologically-inspired and image-computable neural network model. The model combines five key modules: an image feature extractor, bottom-up saliency cues, task-relevant visual features, finite inhibition-of-return, and saccade size constraints. Even though there are no free parameters that are fine-tuned for each specific task, species, or condition, the model produces fixation sequences resembling the universal properties of return fixations. These results provide initial steps towards a mechanistic understanding of the trade-off between rapid foveal recognition and the need to scrutinize previous fixation locations.
- Open Access
- 4 April 2022
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 119 (16)
Face neurons encode nonsemantic features
Significance: Face neurons, which fire more strongly in response to images of faces than to other objects, are a paradigmatic example of object selectivity in the visual cortex. We asked whether such neurons represent the semantic concept of faces or, rather, visual features that are present in faces but do not necessarily count as a face. We created synthetic stimuli that strongly activated face neurons and showed that these stimuli were perceived as clearly distinct from real faces. At the same time, these synthetic stimuli were slightly more often associated with faces than other objects were. These results suggest that so-called face neurons do not represent a semantic category but, rather, represent visual features that correlate with faces.
- 4 April 2022
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 119 (16)
Open AccessFace neurons encode nonsemantic features
Significance: Face neurons, which fire more strongly in response to images of faces than to other objects, are a paradigmatic example of object selectivity in the visual cortex. We asked whether such neurons represent the semantic concept of faces or, rather, visual features that are present in faces but do not necessarily count as a face. We created synthetic stimuli that strongly activated face neurons and showed that these stimuli were perceived as clearly distinct from real faces. At the same time, these synthetic stimuli were slightly more often associated with faces than other objects were. These results suggest that so-called face neurons do not represent a semantic category but, rather, represent visual features that correlate with faces.
- Open Access
Fooling the primate brain with minimal, targeted image manipulation
Artificial neural networks (ANNs) are considered the current best models of biological vision. ANNs are the best predictors of neural activity in the ventral stream; moreover, recent work has demonstrated that ANN models fitted to neuronal activity can guide the synthesis of images that drive pre-specified response patterns in small neuronal populations. Despite the success in predicting and steering firing activity, these results have not been connected with perceptual or behavioral changes. Here we propose an array of methods for creating minimal, targeted image perturbations that lead to changes in both neuronal activity and perception as reflected in behavior. We generated 'deceptive images' of human faces, monkey faces, and noise patterns so that they are perceived as a different, pre-specified target category, and measured both monkey neuronal responses and human behavior to these images. We found several effective methods for changing primate visual categorization that required much smaller image change compared to untargeted noise. Our work shares the same goal with adversarial attack, namely the manipulation of images with minimal, targeted noise that leads ANN models to misclassify the images. Our results represent a valuable step in quantifying and characterizing the differences in perturbation robustness of biological and artificial vision.
Open AccessFooling the primate brain with minimal, targeted image manipulation
Artificial neural networks (ANNs) are considered the current best models of biological vision. ANNs are the best predictors of neural activity in the ventral stream; moreover, recent work has demonstrated that ANN models fitted to neuronal activity can guide the synthesis of images that drive pre-specified response patterns in small neuronal populations. Despite the success in predicting and steering firing activity, these results have not been connected with perceptual or behavioral changes. Here we propose an array of methods for creating minimal, targeted image perturbations that lead to changes in both neuronal activity and perception as reflected in behavior. We generated 'deceptive images' of human faces, monkey faces, and noise patterns so that they are perceived as a different, pre-specified target category, and measured both monkey neuronal responses and human behavior to these images. We found several effective methods for changing primate visual categorization that required much smaller image change compared to untargeted noise. Our work shares the same goal with adversarial attack, namely the manipulation of images with minimal, targeted noise that leads ANN models to misclassify the images. Our results represent a valuable step in quantifying and characterizing the differences in perturbation robustness of biological and artificial vision.
- Open Access
- 15 June 2020
- Published by Public Library of Science (PLoS) in PLOS Computational Biology
- Vol. 16 (6) , e1007973
XDream: Finding preferred stimuli for visual neurons using generative networks and gradient-free optimization
A longstanding question in sensory neuroscience is what types of stimuli drive neurons to fire. The characterization of effective stimuli has traditionally been based on a combination of intuition, insights from previous studies, and luck. A new method termed XDream (EXtending DeepDream with real-time evolution for activation maximization) combined a generative neural network and a genetic algorithm in a closed loop to create strong stimuli for neurons in the macaque visual cortex. Here we extensively and systematically evaluate the performance of XDream. We use ConvNet units as in silico models of neurons, enabling experiments that would be prohibitive with biological neurons. We evaluated how the method compares to brute-force search, and how well the method generalizes to different neurons and processing stages. We also explored design and parameter choices. XDream can efficiently find preferred features for visual units without any prior knowledge about them. XDream extrapolates to different layers, architectures, and developmental regimes, performing better than brute-force search, and often better than exhaustive sampling of >1 million images. Furthermore, XDream is robust to choices of multiple image generators, optimization algorithms, and hyperparameters, suggesting that its performance is locally near-optimal. Lastly, we found no significant advantage to problem-specific parameter tuning. These results establish expectations and provide practical recommendations for using XDream to investigate neural coding in biological preparations. Overall, XDream is an efficient, general, and robust algorithm for uncovering neuronal tuning preferences using a vast and diverse stimulus space. XDream is implemented in Python, released under the MIT License, and works on Linux, Windows, and MacOS.
- 15 June 2020
- Published by Public Library of Science (PLoS) in PLOS Computational Biology
- Vol. 16 (6) , e1007973
Open AccessXDream: Finding preferred stimuli for visual neurons using generative networks and gradient-free optimization
A longstanding question in sensory neuroscience is what types of stimuli drive neurons to fire. The characterization of effective stimuli has traditionally been based on a combination of intuition, insights from previous studies, and luck. A new method termed XDream (EXtending DeepDream with real-time evolution for activation maximization) combined a generative neural network and a genetic algorithm in a closed loop to create strong stimuli for neurons in the macaque visual cortex. Here we extensively and systematically evaluate the performance of XDream. We use ConvNet units as in silico models of neurons, enabling experiments that would be prohibitive with biological neurons. We evaluated how the method compares to brute-force search, and how well the method generalizes to different neurons and processing stages. We also explored design and parameter choices. XDream can efficiently find preferred features for visual units without any prior knowledge about them. XDream extrapolates to different layers, architectures, and developmental regimes, performing better than brute-force search, and often better than exhaustive sampling of >1 million images. Furthermore, XDream is robust to choices of multiple image generators, optimization algorithms, and hyperparameters, suggesting that its performance is locally near-optimal. Lastly, we found no significant advantage to problem-specific parameter tuning. These results establish expectations and provide practical recommendations for using XDream to investigate neural coding in biological preparations. Overall, XDream is an efficient, general, and robust algorithm for uncovering neuronal tuning preferences using a vast and diverse stimulus space. XDream is implemented in Python, released under the MIT License, and works on Linux, Windows, and MacOS.
- Top Cited Papers
Evolving Images for Visual Neurons Using a Deep Generative Network Reveals Coding Principles and Neuronal Preferences