Sonia Poltoratski
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Sonia Poltoratski has a total of 33 co-authors. Here are the top co-authors by number of shared publications:
Frank Tong
Department of Psychology, Vanderbilt University, Nashville, Tennessee 37240, and
9 shared publications- Kalanit Grill-Spector
Stanford University
5 shared publications - Dawn Finzi
Department of Psychology, Stanford University, Stanford, USA
3 shared publications - Jesse Gomez
Princeton Neuroscience Institute, Princeton University
3 shared publications Kalanit Grill-Spector
Department of Psychology, Stanford University
3 shared publicationsSam Ling
Department of Psychological & Brain Sciences, Center for Computational Neuroscience and Neural Technology, Boston University, Boston, Massachusetts
3 shared publications
Latest Publications
- Open Access
- 6 August 2021
- Published by Springer Nature in Nature Communications
- Vol. 12 (1) , 1-13
Holistic face recognition is an emergent phenomenon of spatial processing in face-selective regions
Spatial processing by receptive fields is a core property of the visual system. However, it is unknown how spatial processing in high-level regions contributes to recognition behavior. As face inversion is thought to disrupt typical holistic processing of information in faces, we mapped population receptive fields (pRFs) with upright and inverted faces in the human visual system. Here we show that in face-selective regions, but not primary visual cortex, pRFs and overall visual field coverage are smaller and shifted downward in response to face inversion. From these measurements, we successfully predict the relative behavioral detriment of face inversion at different positions in the visual field. This correspondence between neural measurements and behavior demonstrates how spatial processing in face-selective regions may enable holistic perception. These results not only show that spatial processing in high-level visual regions is dynamically used towards recognition, but also suggest a powerful approach for bridging neural computations by receptive fields to behavior.
- 6 August 2021
- Published by Springer Nature in Nature Communications
- Vol. 12 (1) , 1-13
Open AccessHolistic face recognition is an emergent phenomenon of spatial processing in face-selective regions
Spatial processing by receptive fields is a core property of the visual system. However, it is unknown how spatial processing in high-level regions contributes to recognition behavior. As face inversion is thought to disrupt typical holistic processing of information in faces, we mapped population receptive fields (pRFs) with upright and inverted faces in the human visual system. Here we show that in face-selective regions, but not primary visual cortex, pRFs and overall visual field coverage are smaller and shifted downward in response to face inversion. From these measurements, we successfully predict the relative behavioral detriment of face inversion at different positions in the visual field. This correspondence between neural measurements and behavior demonstrates how spatial processing in face-selective regions may enable holistic perception. These results not only show that spatial processing in high-level visual regions is dynamically used towards recognition, but also suggest a powerful approach for bridging neural computations by receptive fields to behavior.
- Open Access
- 15 April 2021
- Published by Springer Nature in Nature Communications
- Vol. 12 (1) , 1-14
Differential spatial computations in ventral and lateral face-selective regions are scaffolded by structural connections
Face-processing occurs across ventral and lateral visual streams, which are involved in static and dynamic face perception, respectively. However, the nature of spatial computations across streams is unknown. Using functional MRI and population receptive field (pRF) mapping, we measured pRFs in face-selective regions. Results reveal that spatial computations by pRFs in ventral face-selective regions are concentrated around the center of gaze (fovea), but spatial computations in lateral face-selective regions extend peripherally. Diffusion MRI reveals that these differences are mirrored by a preponderance of white matter connections between ventral face-selective regions and foveal early visual cortex (EVC), while connections with lateral regions are distributed more uniformly across EVC eccentricities. These findings suggest a rethinking of spatial computations in face-selective regions, showing that they vary across ventral and lateral streams, and further propose that spatial computations in high-level regions are scaffolded by the fine-grain pattern of white matter connections from EVC.
- 15 April 2021
- Published by Springer Nature in Nature Communications
- Vol. 12 (1) , 1-14
Open AccessDifferential spatial computations in ventral and lateral face-selective regions are scaffolded by structural connections
Face-processing occurs across ventral and lateral visual streams, which are involved in static and dynamic face perception, respectively. However, the nature of spatial computations across streams is unknown. Using functional MRI and population receptive field (pRF) mapping, we measured pRFs in face-selective regions. Results reveal that spatial computations by pRFs in ventral face-selective regions are concentrated around the center of gaze (fovea), but spatial computations in lateral face-selective regions extend peripherally. Diffusion MRI reveals that these differences are mirrored by a preponderance of white matter connections between ventral face-selective regions and foveal early visual cortex (EVC), while connections with lateral regions are distributed more uniformly across EVC eccentricities. These findings suggest a rethinking of spatial computations in face-selective regions, showing that they vary across ventral and lateral streams, and further propose that spatial computations in high-level regions are scaffolded by the fine-grain pattern of white matter connections from EVC.
- Open Access
Holistic face recognition is an emergent phenomenon of spatial integration in face-selective regions
Spatial processing by receptive fields is a core property of the visual system. However, it is unknown how spatial coding in high-level regions contributes to recognition behavior. As face inversion is thought to disrupt typical ‘holistic’ processing of information in faces, we mapped population receptive fields (pRFs) with upright and inverted faces in the human visual system. In face-selective regions, but not primary visual cortex, pRFs and overall visual field coverage were smaller and shifted downward in response to face inversion. From these measurements, we successfully predicted the relative behavioral detriment of face inversion at different positions in the visual field. This correspondence between neural measurements and behavior demonstrates how spatial integration in face-selective regions enables holistic processing. These results not only show that spatial processing in high-level visual regions is dynamically used towards recognition, but also suggest a powerful approach for bridging neural computations by receptive fields to behavior.
Open AccessHolistic face recognition is an emergent phenomenon of spatial integration in face-selective regions
Spatial processing by receptive fields is a core property of the visual system. However, it is unknown how spatial coding in high-level regions contributes to recognition behavior. As face inversion is thought to disrupt typical ‘holistic’ processing of information in faces, we mapped population receptive fields (pRFs) with upright and inverted faces in the human visual system. In face-selective regions, but not primary visual cortex, pRFs and overall visual field coverage were smaller and shifted downward in response to face inversion. From these measurements, we successfully predicted the relative behavioral detriment of face inversion at different positions in the visual field. This correspondence between neural measurements and behavior demonstrates how spatial integration in face-selective regions enables holistic processing. These results not only show that spatial processing in high-level visual regions is dynamically used towards recognition, but also suggest a powerful approach for bridging neural computations by receptive fields to behavior.
- Open Access
Differential spatial computations in ventral and lateral face-selective regions are scaffolded by structural connections
Face-processing occurs across ventral and lateral visual streams, which are involved in static and dynamic face perception, respectively. However, the nature of spatial computations across streams is unknown. Using functional MRI and novel population receptive field (pRF) mapping, we measured pRFs in face-selective regions. Results reveal that spatial computations by pRFs in ventral face-selective regions are concentrated around the center of gaze (fovea), but spatial computations in lateral face-selective regions extend peripherally. Diffusion MRI reveals that these differences are mirrored by a preponderance of white matter connections between ventral face-selective regions and foveal early visual cortex (EVC), while connections with lateral regions are distributed more uniformly across EVC eccentricities. These findings suggest a rethinking of spatial computations in face-selective regions, showing that they vary across ventral and lateral streams, and further propose that spatial computations in high-level regions are scaffolded by the fine-grain pattern of white matter connections from EVC.
Open AccessDifferential spatial computations in ventral and lateral face-selective regions are scaffolded by structural connections
Face-processing occurs across ventral and lateral visual streams, which are involved in static and dynamic face perception, respectively. However, the nature of spatial computations across streams is unknown. Using functional MRI and novel population receptive field (pRF) mapping, we measured pRFs in face-selective regions. Results reveal that spatial computations by pRFs in ventral face-selective regions are concentrated around the center of gaze (fovea), but spatial computations in lateral face-selective regions extend peripherally. Diffusion MRI reveals that these differences are mirrored by a preponderance of white matter connections between ventral face-selective regions and foveal early visual cortex (EVC), while connections with lateral regions are distributed more uniformly across EVC eccentricities. These findings suggest a rethinking of spatial computations in face-selective regions, showing that they vary across ventral and lateral streams, and further propose that spatial computations in high-level regions are scaffolded by the fine-grain pattern of white matter connections from EVC.
- Open Access
Validation and generalization of pixel-wise relevance in convolutional neural networks trained for face classification
The increased use of convolutional neural networks for face recognition in science, governance, and broader society has created an acute need for methods that can show how these 'black box' decisions are made. To be interpretable and useful to humans, such a method should convey a model's learned classification strategy in a way that is robust to random initializations or spurious correlations in input data. To this end, we applied the decompositional pixel-wise attribution method of layer-wise relevance propagation (LRP) to resolve the decisions of several classes of VGG-16 models trained for face recognition. We then quantified how these relevance measures vary with and generalize across key model parameters, such as the pretraining dataset (ImageNet or VGGFace), the finetuning task (gender or identity classification), and random initializations of model weights. Using relevance-based image masking, we find that relevance maps for face classification prove generally stable across random initializations, and can generalize across finetuning tasks. However, there is markedly less generalization across pretraining datasets, indicating that ImageNet- and VGGFace-trained models sample face information differently even as they achieve comparably high classification performance. Fine-grained analyses of relevance maps across models revealed asymmetries in generalization that point to specific benefits of choice parameters, and suggest that it may be possible to find an underlying set of important face image pixels that drive decisions across convolutional neural networks and tasks. Finally, we evaluated model decision weighting against human measures of similarity, providing a novel framework for interpreting face recognition decisions across human and machine.
Open AccessValidation and generalization of pixel-wise relevance in convolutional neural networks trained for face classification
The increased use of convolutional neural networks for face recognition in science, governance, and broader society has created an acute need for methods that can show how these 'black box' decisions are made. To be interpretable and useful to humans, such a method should convey a model's learned classification strategy in a way that is robust to random initializations or spurious correlations in input data. To this end, we applied the decompositional pixel-wise attribution method of layer-wise relevance propagation (LRP) to resolve the decisions of several classes of VGG-16 models trained for face recognition. We then quantified how these relevance measures vary with and generalize across key model parameters, such as the pretraining dataset (ImageNet or VGGFace), the finetuning task (gender or identity classification), and random initializations of model weights. Using relevance-based image masking, we find that relevance maps for face classification prove generally stable across random initializations, and can generalize across finetuning tasks. However, there is markedly less generalization across pretraining datasets, indicating that ImageNet- and VGGFace-trained models sample face information differently even as they achieve comparably high classification performance. Fine-grained analyses of relevance maps across models revealed asymmetries in generalization that point to specific benefits of choice parameters, and suggest that it may be possible to find an underlying set of important face image pixels that drive decisions across convolutional neural networks and tasks. Finally, we evaluated model decision weighting against human measures of similarity, providing a novel framework for interpreting face recognition decisions across human and machine.
- Open Access
- 5 March 2020
- Published by Society for Neuroscience in The Journal of Neuroscience
- Vol. 40 (16) , 3292-3303
Resolving the Spatial Profile of Figure Enhancement in Human V1 through Population Receptive Field Modeling
The detection and segmentation of meaningful figures from their background is one of the primary functions of vision. While work in nonhuman primates has implicated early visual mechanisms in this figure–ground modulation, neuroimaging in humans has instead largely ascribed the processing of figures and objects to higher stages of the visual hierarchy. Here, we used high-field fMRI at 7 Tesla to measure BOLD responses to task-irrelevant orientation-defined figures in human early visual cortex (N= 6, four females). We used a novel population receptive field mapping-based approach to resolve the spatial profiles of two constituent mechanisms of figure–ground modulation: a local boundary response, and a further enhancement spanning the full extent of the figure region that is driven by global differences in features. Reconstructing the distinct spatial profiles of these effects reveals that figure enhancement modulates responses in human early visual cortex in a manner consistent with a mechanism of automatic, contextually driven feedback from higher visual areas.SIGNIFICANCE STATEMENTA core function of the visual system is to parse complex 2D input into meaningful figures. We do so constantly and seamlessly, both by processing information about visible edges and by analyzing large-scale differences between figure and background. While influential neurophysiology work has characterized an intriguing mechanism that enhances V1 responses to perceptual figures, we have a poor understanding of how the early visual system contributes to figure–ground processing in humans. Here, we use advanced computational analysis methods and high-field human fMRI data to resolve the distinct spatial profiles of local edge and global figure enhancement in the early visual system (V1 and LGN); the latter is distinct and consistent with a mechanism of automatic, stimulus-driven feedback from higher-level visual areas.
- 5 March 2020
- Published by Society for Neuroscience in The Journal of Neuroscience
- Vol. 40 (16) , 3292-3303
Open AccessResolving the Spatial Profile of Figure Enhancement in Human V1 through Population Receptive Field Modeling
The detection and segmentation of meaningful figures from their background is one of the primary functions of vision. While work in nonhuman primates has implicated early visual mechanisms in this figure–ground modulation, neuroimaging in humans has instead largely ascribed the processing of figures and objects to higher stages of the visual hierarchy. Here, we used high-field fMRI at 7 Tesla to measure BOLD responses to task-irrelevant orientation-defined figures in human early visual cortex (N= 6, four females). We used a novel population receptive field mapping-based approach to resolve the spatial profiles of two constituent mechanisms of figure–ground modulation: a local boundary response, and a further enhancement spanning the full extent of the figure region that is driven by global differences in features. Reconstructing the distinct spatial profiles of these effects reveals that figure enhancement modulates responses in human early visual cortex in a manner consistent with a mechanism of automatic, contextually driven feedback from higher visual areas.SIGNIFICANCE STATEMENTA core function of the visual system is to parse complex 2D input into meaningful figures. We do so constantly and seamlessly, both by processing information about visible edges and by analyzing large-scale differences between figure and background. While influential neurophysiology work has characterized an intriguing mechanism that enhances V1 responses to perceptual figures, we have a poor understanding of how the early visual system contributes to figure–ground processing in humans. Here, we use advanced computational analysis methods and high-field human fMRI data to resolve the distinct spatial profiles of local edge and global figure enhancement in the early visual system (V1 and LGN); the latter is distinct and consistent with a mechanism of automatic, stimulus-driven feedback from higher-level visual areas.
- Open Access
- 19 December 2019
- Published by Cold Spring Harbor Laboratory in bioRxiv
Resolving the spatial profile of figure enhancement in human V1 through population receptive field modeling
The detection and segmentation of meaningful figures from their background is a core function of vision. While work in non-human primates has implicated early visual mechanisms in this figure-ground modulation, neuroimaging in humans has instead largely ascribed the processing of figures and objects to higher stages of the visual hierarchy. Here, we used high-field fMRI at 7Tesla to measure BOLD responses to task-irrelevant orientation-defined figures in human early visual cortex, and employed a novel population receptive field (pRF) mapping-based approach to resolve the spatial profiles of two constituent mechanisms of figure-ground modulation: a local boundary response, and a further enhancement spanning the full extent of the figure region that is driven by global differences in features. Reconstructing the distinct spatial profiles of these effects reveals that figure enhancement modulates responses in human early visual cortex in a manner consistent with a mechanism of automatic, contextually-driven feedback from higher visual areas.Significance Statement: A core function of the visual system is to parse complex 2D input into meaningful figures. We do so constantly and seamlessly, both by processing information about visible edges and by analyzing large-scale differences between figures and background. While influential neurophysiology work has characterized an intriguing mechanism that enhances V1 responses to perceptual figures, we have a poor understanding of how the early visual system contributes to figure-ground processing in humans. Here, we use advanced computational analysis methods and high-field human fMRI data to resolve the distinct spatial profiles of local edge and global figure enhancement in the early visual system (V1 and LGN); the latter is distinct and consistent a mechanism of automatic, stimulus-driven feedback from higher-level visual areas.
- 19 December 2019
- Published by Cold Spring Harbor Laboratory in bioRxiv
Open AccessResolving the spatial profile of figure enhancement in human V1 through population receptive field modeling
The detection and segmentation of meaningful figures from their background is a core function of vision. While work in non-human primates has implicated early visual mechanisms in this figure-ground modulation, neuroimaging in humans has instead largely ascribed the processing of figures and objects to higher stages of the visual hierarchy. Here, we used high-field fMRI at 7Tesla to measure BOLD responses to task-irrelevant orientation-defined figures in human early visual cortex, and employed a novel population receptive field (pRF) mapping-based approach to resolve the spatial profiles of two constituent mechanisms of figure-ground modulation: a local boundary response, and a further enhancement spanning the full extent of the figure region that is driven by global differences in features. Reconstructing the distinct spatial profiles of these effects reveals that figure enhancement modulates responses in human early visual cortex in a manner consistent with a mechanism of automatic, contextually-driven feedback from higher visual areas.Significance Statement: A core function of the visual system is to parse complex 2D input into meaningful figures. We do so constantly and seamlessly, both by processing information about visible edges and by analyzing large-scale differences between figures and background. While influential neurophysiology work has characterized an intriguing mechanism that enhances V1 responses to perceptual figures, we have a poor understanding of how the early visual system contributes to figure-ground processing in humans. Here, we use advanced computational analysis methods and high-field human fMRI data to resolve the distinct spatial profiles of local edge and global figure enhancement in the early visual system (V1 and LGN); the latter is distinct and consistent a mechanism of automatic, stimulus-driven feedback from higher-level visual areas.
- Open Access
- 6 September 2019
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 19 (10) , 258c
Population receptive field measurements of stimulus-driven effects in face-selective areas
Classic theories suggest that high-level visual recognition is largely invariant to the item’s position and size, but this is challenged by consistent reports of spatial biases and position information in high-level visual regions. Recent work in our lab (Kay, Weiner, and Grill-Spector, 2015; Gomez et al. 2018) has used population receptive field (pRF) models to computationally quantify spatial representations in face selective areas. However, it remains unknown how these spatial representations may contribute to face recognition behavior. Spatial integration of information is critical to two behavioral hallmarks of face processing: holistic processing, and competition between multiple items. In two experiments (Fig 1), we used 3T fMRI to map pRFs in 6 participants with stimuli thought to disrupt spatial processing enroute to face recognition: inverted faces (Experiment 1), and face pairs (Experiment 2), and compared these pRF properties to those measured with single, upright faces. To ensure fixation, participants performed a 1-back task on rapidly presented letters. Compared to upright faces, inverted faces yielded (1) a shift in the location of pRFs centers to the lower visual field, (2) lower variance explained by the pRF model, and (3) lower response amplitude (Fig 2). Further, these differences increased in magnitude from IOG-, to pFus-, to mFus-faces. Meanwhile, compared to a single upright face, presenting a face pair reduced pRF size and coverage across manipulations of the size of the single face, as predicted by classic theories of visual competition (Fig 3). In both experiments, we did not observe differences in pRFs or visual field coverage in V1 across varying mapping stimuli. Together, these results show for the first time that not only top-down attention, but also bottom-up stimulus driven properties can shift pRFs in face-selective regions. These data suggest that pRFs in high-level regions have complex properties that may constrain recognition behavior.
- 6 September 2019
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 19 (10) , 258c
Open AccessPopulation receptive field measurements of stimulus-driven effects in face-selective areas
Classic theories suggest that high-level visual recognition is largely invariant to the item’s position and size, but this is challenged by consistent reports of spatial biases and position information in high-level visual regions. Recent work in our lab (Kay, Weiner, and Grill-Spector, 2015; Gomez et al. 2018) has used population receptive field (pRF) models to computationally quantify spatial representations in face selective areas. However, it remains unknown how these spatial representations may contribute to face recognition behavior. Spatial integration of information is critical to two behavioral hallmarks of face processing: holistic processing, and competition between multiple items. In two experiments (Fig 1), we used 3T fMRI to map pRFs in 6 participants with stimuli thought to disrupt spatial processing enroute to face recognition: inverted faces (Experiment 1), and face pairs (Experiment 2), and compared these pRF properties to those measured with single, upright faces. To ensure fixation, participants performed a 1-back task on rapidly presented letters. Compared to upright faces, inverted faces yielded (1) a shift in the location of pRFs centers to the lower visual field, (2) lower variance explained by the pRF model, and (3) lower response amplitude (Fig 2). Further, these differences increased in magnitude from IOG-, to pFus-, to mFus-faces. Meanwhile, compared to a single upright face, presenting a face pair reduced pRF size and coverage across manipulations of the size of the single face, as predicted by classic theories of visual competition (Fig 3). In both experiments, we did not observe differences in pRFs or visual field coverage in V1 across varying mapping stimuli. Together, these results show for the first time that not only top-down attention, but also bottom-up stimulus driven properties can shift pRFs in face-selective regions. These data suggest that pRFs in high-level regions have complex properties that may constrain recognition behavior.
- 1 June 2019
- Published by Elsevier in Current Biology
- Vol. 29 (12) , 2051-2057.e3
Figure-Ground Modulation in the Human Lateral Geniculate Nucleus Is Distinguishable from Top-Down Attention
- 1 June 2019
- Published by Elsevier in Current Biology
- Vol. 29 (12) , 2051-2057.e3
Figure-Ground Modulation in the Human Lateral Geniculate Nucleus Is Distinguishable from Top-Down Attention
- Open Access
- 1 September 2018
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 18 (10) , 25
Cortical feedback mediates figure-ground modulation in the human lateral geniculate nucleus
The lateral geniculate nucleus (LGN) is the earliest site of the visual hierarchy that receives top-down feedback, yet the functional role of this feedback is poorly understood. However, growing research suggests that the human LGN is involved in more sophisticated visual and cognitive processes, showing modulation by covert attention (O'Connor et al., 2002) and evidence of orientation-selective processing (Ling et al. 2015). Here, we show that perceptual figures elicit an automatic form of feedback modulation that propagates from the binocular visual cortex to the LGN. This stimulus-driven feedback leads to the enhancement of figures in the LGN even in the absence of directed attention. Using high-resolution fMRI at 7 Tesla to record human brain activity, we first measured fMRI responses to orientation-defined figures presented to the left and right of fixation, cuing participants to spatially attend to one figure while ignoring the other. Spatial attention led to enhanced responses in the LGN, consistent with prior work, but more importantly, orientation-defined figures produced elevated responses even when the figure was unattended. In a second experiment, we manipulated whether the figure and the surround stimuli were presented to the same eye or to different eyes. This design leverages the binocular organization of the early visual system: V1 is considered the first stage along the visual hierarchy in which signals from the two eyes are strongly integrated. Nevertheless, we found that the LGN was reliably modulated when figure and ground were presented to different eyes, implicating a mechanism top-down feedback from binocular cortical neurons for figure-ground modulation in the LGN. Meeting abstract presented at VSS 2018
- 1 September 2018
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 18 (10) , 25
Open AccessCortical feedback mediates figure-ground modulation in the human lateral geniculate nucleus
The lateral geniculate nucleus (LGN) is the earliest site of the visual hierarchy that receives top-down feedback, yet the functional role of this feedback is poorly understood. However, growing research suggests that the human LGN is involved in more sophisticated visual and cognitive processes, showing modulation by covert attention (O'Connor et al., 2002) and evidence of orientation-selective processing (Ling et al. 2015). Here, we show that perceptual figures elicit an automatic form of feedback modulation that propagates from the binocular visual cortex to the LGN. This stimulus-driven feedback leads to the enhancement of figures in the LGN even in the absence of directed attention. Using high-resolution fMRI at 7 Tesla to record human brain activity, we first measured fMRI responses to orientation-defined figures presented to the left and right of fixation, cuing participants to spatially attend to one figure while ignoring the other. Spatial attention led to enhanced responses in the LGN, consistent with prior work, but more importantly, orientation-defined figures produced elevated responses even when the figure was unattended. In a second experiment, we manipulated whether the figure and the surround stimuli were presented to the same eye or to different eyes. This design leverages the binocular organization of the early visual system: V1 is considered the first stage along the visual hierarchy in which signals from the two eyes are strongly integrated. Nevertheless, we found that the LGN was reliably modulated when figure and ground were presented to different eyes, implicating a mechanism top-down feedback from binocular cortical neurons for figure-ground modulation in the LGN. Meeting abstract presented at VSS 2018