Jenny M. Bosten

We exploited co-occurrences between color and other properties of natural scenes to identify and visualize functionally distinct brain regions. For each voxel in the Natural Scenes Dataset (NSD), we computed a scaled response-weighted average of the stimulus images. The colors of “voxel-preferred images” (VPIs) reflect stimulus properties that covary with color in natural scenes: Color serves as a tag for functional distinctions in voxel responses. Mapping VPIs onto cortical surfaces revealed reliable and structured color patterns that segment voxel clusters. Boundaries between clusters of similarly colored VPIs tend to coincide with boundaries defined using other methods, and heterogeneity within regions suggests functional subdivisions. VPIs provide a simple data-driven method for analyzing fMRI responses to natural scenes and visualizing cortical organization.

Color information is thought to enter the cortex via two dominant retinogeniculate pathways, one signaling teal to red, and the other violet to lime color variation. The cortex is thought to transform this representation, but the properties of human cortical color mechanisms are not very well understood. In four experiments, we characterized the tuning of cortical color mechanisms by measuring the intermodulation of steady-state visually evoked potentials (SSVEPs), thought to index the extent to which shared neural resources process stimuli flickering at different frequencies. Stimuli were isoluminant chromatic checkerboards where odd and even checks flickered at different frequencies. As hue dissimilarity between odd and even checks increased, the amplitude of an intermodulation component (I1) at the sum of the two stimulus frequencies decreased, revealing color tuning functions. In Experiment 1, we found similar broad tuning functions for “cardinal” and intermediate color axes, implying the action of intermediately tuned cortical color mechanisms. In Experiment 2 we found similar tuning functions for “checkerboards” without perceptible edges because the checks were formed from single pixels (~0.096°), implying that the underlying neural populations do not rely on spatial chromatic edges. In Experiment 3 we found consistent color tuning functions across check sizes. In Experiment 4 we measured full 360° tuning functions and found results compatible with opponent color responses. The observed cortical color tuning functions are consistent with those measured using psychophysics and electrophysiology, implying that the method is useful for investigating color representations in the brain.

Convolutional neural networks (CNNs) trained for image categorization are known to have color-selective units that are tuned to particular colors. Analogously to the analysis of color representations in humans using brain imaging data, the representation of color within layers of a trained network can be characterized by constructing representational dissimilarity matrices (RDMs) and by using multidimensional scaling (MDS) to visualize geometric representational color spaces. Human color representations show flexibility dependent on the task. We trained CNNs on a set of simple chromatic stimuli but varied the “task” to require either color categorization, an analog of a color appearance rating, or luminance or spatial judgments that may not require color at all. We found that color representations within trained networks differed reliably and distinctively between task conditions, and that structured representations developed for color-relevant training conditions that were appropriate to the task. Color representations for different task conditions diverged through network layers toward the output layer, but they were significantly different even for layer 2 near the input layer. The variance between network instances was lowest for color-relevant tasks. For two of the tasks initially assumed to be “color-irrelevant,” reliable and distinctive color representations developed. For these tasks requiring luminance or spatial judgments, color must provide a useful cue even though it is not required in the network output. These results or similar models may be used to generate hypotheses for how color representations in the human brain vary with task (whether tasks obviously require color or not), which could be tested using neuroimaging.

Characterization of postreceptoral chromatic mechanisms is an active challenge in color vision research. Adding to evidence from human psychophysics and primate neurophysiology, human neuroimaging using electroencephalography (EEG) in combination with steady-state visually evoked potentials (SSVEPs) has recently been used to investigate the tuning of cortical color mechanisms. We compare the performance of two SSVEP-based methods for measuring the tuning of cortical color mechanisms: an intermodulation component (IM)-based method and an oddball method. Our results support SSVEPs as an effective tool for investigating the tuning of cortical color mechanisms. The intermodulation method is efficient and provides high signal-to-noise ratios, but the oddball method can also yield clear tuning functions if multiple SSVEP harmonics are analyzed in combination.

We propose a standardized naming system for vertebrate visual photoreceptors (i.e., rods and cones) that reflects our current understanding of their evolutionary history. Vertebrate photoreceptors have been studied for well over a century, but a fixed nomenclature for referring to orthologous cell types across diverse species has been lacking. Instead, photoreceptors have been variably - and often confusingly - named according to morphology, presence/absence of ‘rhodopsin,’ spectral sensitivity, chromophore usage, and/or the gene family of the opsin(s) they express. Here, we propose a unified nomenclature for vertebrate rods and cones that aligns with the naming systems of other retinal cell classes and that is based on the photoreceptor’s putative ancestral derivation. This classification is informed by the functional, anatomical, developmental and molecular identities of the neuron as a whole, including the expression of deeply conserved transcription factors required for development. The proposed names will be applicable across all vertebrates and indicative of the widest-possible range of properties, including their postsynaptic wiring, and hence will allude to their common and species-specific roles in vision. Furthermore, the naming system is open-ended to accommodate the future discovery of as-yet unknown photoreceptor types.

To what extent is perception shaped by low-level statistical regularities of our visual environments and on what time scales? We characterized the chromatic ‘visual diets’ of people living in remote rainforest and urban environments, using calibrated head-mounted cameras worn by participants as they went about their daily lives. All environments had chromatic distributions with the most variance along a blue–yellow axis, but the extent of this bias differed across locations. If colour perception is calibrated to the visual environments in which participants are immersed, variation in the extent of the bias in scene statistics should have a corresponding impact on perceptual judgements. To test this, we measured colour discrimination and preferences for distributions of colour for people living in different environments. Group differences in the extent of blue–yellow bias in colour discrimination were consistent with perceptual learning in local environments. Preferences for colour distributions aligned with scene statistics, but not specifically to local environments, and one group preferred distributions along an unnatural colour axis orthogonal to that dominant in natural scenes. Our study shows the benefits of conducting psychophysics with people at remote locations for understanding the commonalities and diversity in human perception.

To what extent are perception and aesthetics shaped by low-level statistical regularities of our visual environments, and on what time scales? We characterized the chromatic statistics of the ‘visual diets’ of people living in remote rainforest and urban environments, by randomly sampling images using calibrated head-mounted cameras as participants went about daily lives. All environments, as is typical for natural scenes, had chromatic distributions with most variance along a roughly blue-yellow axis, but the extent of this bias differed across locations. If color perception and aesthetics are calibrated to the visual environments in which participants are currently immersed, variation in the extent of the bias in scene statistics should have a corresponding impact on perception and aesthetic judgements. To test this, we measured color discrimination and preferences for distributions of color for people living in the different environments. Group differences in the extent of blue-yellow bias in color discrimination were in the opposite direction to that predicted by the differences in scene statistics under an efficient coding framework, but were consistent with perceptual learning in local environments. Preferences for color distributions aligned with scene statistics, but not specifically to local environments, and one group preferred color distributions along an unnatural color axis orthogonal to that dominant in natural scenes. These preferences could be explained by an interaction of culture and perceptual fluency. Our study shows the benefits of conducting psychophysics with people at remote locations for understanding the commonalities and diversity in human perception.