Meredith N. Schmehl

We find that the inferior colliculus, a primarily auditory brain area, displays distinct population-level responses to visual stimuli. We also find that visual cues can influence the auditory responses of individual neurons. Together, the results provide insight into how relatively early sensory areas may play a role in combining multiple sensory modalities to refine the perception of complex environments.

A recently proposed theory of neural representation postulates that when more than one stimulus is to be encoded, neurons in that representation may fluctuate between encoding each item. To date, such work has considered only cases in which both stimuli are of the same sensory modality, such as how two sounds are encoded in an auditory brain region. Here, we consider the influence of sensory modality on this process, asking whether stimuli of a non-dominant sensory modality can evoke or influence the fluctuating activity patterns observed for combinations of stimuli of a dominant sensory modality. Specifically, we investigated the impact of combining visual and auditory stimuli on single-unit activity in the inferior colliculus, a predominantly auditory structure, in monkeys performing a behavioral task involving reporting all stimulus locations in a given trial. We found that fluctuating activity is evident when a single sound and a single visual stimulus are paired at the same spatial location, indicating that fluctuating activity can be induced when one item is not from a brain area’s preferred modality, and even when only a single spatial location is involved and spatial attention is directed to only one place. Fluctuating activity was also identified when visual stimuli were paired with sounds presented at two spatial locations. The IC appeared to maintain a representation of all stimuli, regardless of the sequence in which the monkey localized the targets (although in one monkey there was a bias toward greater representation of the second reported location). Together, these results indicate that fluctuating activity occurs in response to multimodal stimuli in a predominantly unimodal brain structure and is geared toward preservation of all sensory information rather than attentional filtering. Impact Statement We found that when visual stimuli are paired with sounds, they can induce fluctuating activity in neurons that are primarily sensitive to sound alone. Such activity fluctuations may permit the representation of both the visual and auditory stimuli in an overlapping population of neurons.

How multisensory cues affect processing in early sensory brain areas is not well understood. The inferior colliculus (IC) is an early auditory structure that is visually responsive (Porter et al., 2007; Bulkin & Groh, 2012b, 2012a), but little is known about how visual signals affect the IC’s auditory representation. We explored how visual cues affect both spiking and local field potential (LFP) activity in the IC of two monkeys performing a task involving saccades to auditory, visual, or combined audiovisual stimuli. We confirm that LFPs are sensitive to the onset of fixation lights as well as the onset of visual targets presented during steady fixation. The LFP waveforms evoked by combined audiovisual stimuli differed from those evoked by sounds alone. In single-unit spiking activity, responses were weak when visual stimuli were presented alone, but visual stimuli could modulate the activity evoked by sounds in a stronger way. Such modulations could involve either increases or decreases in activity, and whether increases or decreases were observed was variable and not obviously correlated with the responses evoked by visual or auditory stimuli alone. These findings indicate that visual stimuli shape the IC’s auditory representation in flexible ways that differ from those observed previously in multisensory areas. Significance Statement We find that the inferior colliculus, a primarily auditory brain area, displays distinct population-level responses to visual stimuli. We also find that visual cues can influence the auditory responses of individual neurons. Together, the results provide insight into how relatively early sensory areas may play a role in combining multiple sensory modalities to refine the perception of complex environments.

How neural representations preserve information about multiple stimuli is mysterious. Because tuning of individual neurons is coarse (e.g., visual receptive field diameters can exceed perceptual resolution), the populations of neurons potentially responsive to each individual stimulus can overlap, raising the question of how information about each item might be segregated and preserved in the population. We recently reported evidence for a potential solution to this problem: when two stimuli were present, some neurons in the macaque visual cortical areas V1 and V4 exhibited fluctuating firing patterns, as if they responded to only one individual stimulus at a time (Jun et al., 2022). However, whether such an information encoding strategy is ubiquitous in the visual pathway and thus could constitute a general phenomenon remains unknown. Here, we provide new evidence that such fluctuating activity is also evoked by multiple stimuli in visual areas responsible for processing visual motion (middle temporal visual area, MT), and faces (middle fundus and anterolateral face patches in inferotemporal cortex – areas MF and AL), thus extending the scope of circumstances in which fluctuating activity is observed. Furthermore, consistent with our previous results in the early visual area V1, MT exhibits fluctuations between the representations of two stimuli when these form distinguishable objects but not when they fuse into one perceived object, suggesting that fluctuating activity patterns may underlie visual object formation. Taken together, these findings point toward an updated model of how the brain preserves sensory information about multiple stimuli for subsequent processing and behavioral action.

How neural representations preserve information about multiple stimuli is mysterious. Because tuning of individual neurons is coarse (for example, visual receptive field diameters can exceed perceptual resolution), the populations of neurons potentially responsive to each individual stimulus can overlap, raising the question of how information about each item might be segregated and preserved in the population. We recently reported evidence for a potential solution to this problem: when two stimuli were present, some neurons in the macaque visual cortical areas V1 and V4 exhibited fluctuating firing patterns, as if they responded to only one individual stimulus at a time. However, whether such an information encoding strategy is ubiquitous in the visual pathway and thus could constitute a general phenomenon remains unknown. Here we provide new evidence that such fluctuating activity is also evoked by multiple stimuli in visual areas responsible for processing visual motion (middle temporal visual area, MT), and faces (middle fundus and anterolateral face patches in inferotemporal cortex – areas MF and AL), thus extending the scope of circumstances in which fluctuating activity is observed. Furthermore, consistent with our previous results in the early visual area V1, MT exhibits fluctuations between the representations of two stimuli when these form distinguishable objects but not when they fuse into one perceived object, suggesting that fluctuating activity patterns may underlie visual object formation. Taken together, these findings point toward an updated model of how the brain preserves sensory information about multiple stimuli for subsequent processing and behavioral action. Impact Statement: We find neural fluctuations in multiple areas along the visual cortical hierarchy that could allow the brain to represent distinct co-occurring visual stimuli.

As new challenges arise in the 21st century, state and local governments play an increasingly critical role in science policy, contrasting the traditional focus on the federal government in this landscape. To meet these challenges, states require access to subject area expertise and evidence-based advisory resources as part of their policy toolkits. Many states have independent academies of science that have potential to provide scientific expertise to state governments. However, steps need to be taken to capitalize on these resources and integrate them with other key elements in the policymaking process. By prioritizing the development of relationships with state and local governments, academies of science and other state-level scientific entities could improve the utility of their advisory resources. We present case studies from Connecticut and Missouri, where such a model has allowed scientists to contribute to policymaking on state-level issues. We further discuss the benefits and limitations of this advisory model and explain how this approach can benefit states with different political compositions and legislative structures. By partnering more intentionally with state and local governments, academies of science can make more effective contributions to address the growing science policy issues of the 21st century and beyond.

Coordination between different sensory systems is a necessary element of sensory processing. Where and how signals from different sense organs converge onto common neural circuitry have become topics of increasing interest in recent years. In this article, we focus specifically on visual–auditory interactions in areas of the mammalian brain that are commonly considered to be auditory in function. The auditory cortex and inferior colliculus are two key points of entry where visual signals reach the auditory pathway, and both contain visual- and/or eye movement–related signals in humans and other animals. The visual signals observed in these auditory structures reflect a mixture of visual modulation of auditory-evoked activity and visually driven responses that are selective for stimulus location or features. These key response attributes also appear in the classic visual pathway but may play a different role in the auditory pathway: to modify auditory rather than visual perception. Finally, while this review focuses on two particular areas of the auditory pathway where this question has been studied, robust descending as well as ascending connections within this pathway suggest that undiscovered visual signals may be present at other stages as well.

COVID-19 has renewed the importance of science diplomacy, with international cooperation and scientific collaboration across borders more critical than ever to combat the pandemic. To better disseminate rapidly developing innovations and scientific expertise, there is a need to reevaluate the relationship between modes of scientific communications and traditional diplomatic channels. Although science diplomacy is traditionally between national governments or international organizations such as the United Nations, individual connections between scientists from different countries is itself a form of science diplomacy. Scientific conferences are a key component of science diplomacy and more broadly bring scientists and their industrial, governmental, and philanthropic partners together from across the world to discuss major scientific advances and challenges. Through international scientific collaborations that will form during such conferences, the scientific community can broker new partnerships where formal political relationships are strained. Thus, it is essential that virtual scientific conferences are made accessible in order to facilitate effective science diplomacy. Here we explore the major obstacles of in-person conferences (high cost, locally-focused, and less equitable) which limit how global conversations can benefit from scientific knowledge, as well as opportunities offered by virtual meetings (affordability, global participation, equitable nature) to foster science diplomacy. Further, we provide recommendations that can enable future conferences, both fully or partially virtual, to strengthen science diplomacy post-COVID-19.