Brian J. White

The tremendous increase in the use of video-based eye tracking has made it possible to collect eye tracking data from thousands of participants. The traditional procedures for the manual detection and classification of saccades and for trial categorization (e.g., correct vs. incorrect) are not viable for the large datasets being collected. Additionally, video-based eye trackers allow for the analysis of pupil responses and blink behaviors. Here, we present a detailed description of our pipeline for collecting, storing, and cleaning data, as well as for organizing participant codes, which are fairly lab-specific but nonetheless, are important precursory steps in establishing standardized pipelines. More importantly, we also include descriptions of the automated detection and classification of saccades, blinks, “blincades” (blinks occurring during saccades), and boomerang saccades (two nearly simultaneous saccades in opposite directions where speed-based algorithms fail to split them), This is almost entirely task-agnostic and can be used on a wide variety of data. We additionally describe novel findings regarding post-saccadic oscillations and provide a method to achieve more accurate estimates for saccade end points. Lastly, we describe the automated behavior classification for the interleaved pro/anti-saccade task (IPAST), a task that probes voluntary and inhibitory control. This pipeline was evaluated using data collected from 592 human participants between 5 and 93 years of age, making it robust enough to handle large clinical patient datasets. In summary, this pipeline has been optimized to consistently handle large datasets obtained from diverse study cohorts (i.e., developmental, aging, clinical) and collected across multiple laboratory sites.

Spontaneous eye blinking is gaining popularity as a proxy for higher cognitive functions, as it is readily modulated by both environmental demands and internal processes. Prior studies were impoverished in sample size, sex representation, and age distribution, making it difficult to establish a complete picture of the behavior. Here we present eye-tracking data from a large cohort of normative participants (n = 604; 393 F; aged 5–93 years) performing two tasks: one with structured, discrete trials (interleaved pro-/anti-saccade task, IPAST) and one with a less structured, continuous organization in which participants watch movies (free-viewing; FV). Sex- and age-based analyses revealed that females had higher blink rates between the ages of 22 and 58 years in the IPAST and 22 and 34 years in FV. We derived a continuous measure of blink probability to reveal behavioral changes driven by stimulus appearance in both paradigms. In the IPAST, blinks were suppressed near stimulus appearance, particularly on correct anti-saccade trials, which we attribute to the stronger inhibitory control required for anti-saccades compared with pro-saccades. In FV, blink suppression occurred immediately after scene changes, and the effect was sustained on scenes where gaze clustered among participants (indicating engagement of attention). Females were more likely than males to blink during appearance of novel stimuli in both tasks, but only within the age bin of 18–44 years. The consistency of blink patterns in each paradigm endorses blinking as a sensitive index for changes in visual processing and attention, while sex and age differences drive interindividual variability.

The superior colliculus (SC) is a multilayered midbrain structure with depth-dependent cortical/subcortical connectivity, and a longstanding role in the control of attention/gaze. While the superficial layers (SCs) have been associated with a bottom-up saliency map, the intermediate layers (SCi) have been described as a priority map, where neuronal signals related to visual salience and behavioral relevance combine to determine attention/gaze. However, the use of single electrodes to understand SC laminar function has been a major limitation due to inaccurate depth estimates. Here, we examined depth-dependent processing of stimuli of different salience/relevance across the intermediate and deeper SC layers using a linear microelectrode (LMA; 16ch, 200 μm inter-contact spacing). Rhesus monkeys were presented with an array of oriented color stimuli (~200 items) with two salient but feature-distinct oddballs. One oddball was goal-relevant (salient/relevant), the other goal-irrelevant (salient/non-relevant), and both were embedded in a feature-homogenous array of ‘distractors’ (non-salient/non-relevant). Following array onset, the animals maintained fixation for 0.5–0.7s allowing temporal separation between visual- and saccade- processes. The fixation stimulus then disappeared, instructing the animal to saccade to the goal-relevant oddball for a reward. We examined multiunit activity (MUA), local field potential activity (LFP), and current source density (CSD). We observed a depth-specific change from net inward-to-outward current flow in the saccade-evoked LFP, and corresponding CSD, indicating a depth-dependent functional distinction. We also observed depth-dependent oddball selectivity corresponding roughly to the upper window (~1600 μm) defined by the CSD cutoff, consistent with SCi. This oddball response was maximal at the center of the window and systematically attenuated dorsal and ventral from this. The deepest sites showed visual and saccadic responses, yet were not oddball selective. These results are consistent with a depth-dependent priority map in the SCi that combines information about stimulus saliency and relevancy to systematically rank order map locations for attention/gaze control.

The saliency map has played a long‐standing role in models and theories of visual attention, and it is now supported by neurobiological evidence from several cortical and subcortical brain areas. While visual saliency is computed during moments of active fixation, it is not known whether the same is true while engaged in smooth pursuit of a moving stimulus, which is very common in real‐world vision. Here, we examined extrafoveal saliency coding in the superior colliculus, a midbrain area associated with attention and gaze, during smooth pursuit eye movements. We found that SC neurons from the superficial visual layers showed a robust representation of peripheral saliency evoked by a conspicuous stimulus embedded in a wide‐field array of goal‐irrelevant stimuli. In contrast, visuomotor neurons from the intermediate saccade‐related layers showed a poor saliency representation, even though most of these neurons were visually responsive during smooth pursuit. These results confirm and extend previous findings that place the SCs in a unique role as a saliency map that monitors peripheral vision during foveation of stationary and now moving objects.

Significance: Theories of visual attention postulate the existence of a saliency map that guides attention/gaze toward the most visually conspicuous stimuli in complex scenes. This study compared saliency coding in the two dominant visual gateways: the primary visual cortex (V1) and the evolutionarily older visual system that exists in the midbrain superior colliculus. Our results show that neurons in the superficial visual layers of the superior colliculus (SCs) encoded saliency earlier and more robustly than V1 neurons. This was surprising, because the dominant input to the SCs arises from V1. This result is in line with models that place a feature processing stage (V1) before the feature-agnostic saliency map in SCs.

Models of visual attention postulate the existence of a saliency map whose function is to guide attention and gaze to the most conspicuous regions in a visual scene. Although cortical representations of saliency have been reported, there is mounting evidence for a subcortical saliency mechanism, which pre-dates the evolution of neocortex. Here, we conduct a strong test of the saliency hypothesis by comparing the output of a well-established computational saliency model with the activation of neurons in the primate superior colliculus (SC), a midbrain structure associated with attention and gaze, while monkeys watched video of natural scenes. We find that the activity of SC superficial visual-layer neurons (SCs), specifically, is well-predicted by the model. This saliency representation is unlikely to be inherited from fronto-parietal cortices, which do not project to SCs, but may be computed in SCs and relayed to other areas via tectothalamic pathways.

The mechanisms that underlie the integration of visual and goal-related signals for the production of saccades remain poorly understood. Here, we examined how spatial proximity of competing stimuli shapes goal-directed responses in the superior colliculus (SC), a midbrain structure closely associated with the control of visual attention and eye movements. Monkeys were trained to perform an oculomotor-capture task [Theeuwes, J., Kramer, A. F., Hahn, S., Irwin, D. E., & Zelinsky, G. J. Influence of attentional capture on oculomotor control. Journal of Experimental Psychology. Human Perception and Performance, 25, 1595–1608, 1999], in which a target singleton was revealed via an isoluminant color change in all but one item. On a portion of the trials, an additional salient item abruptly appeared near or far from the target. We quantified how spatial proximity between the abrupt-onset and the target shaped the goal-directed response. We found that the appearance of an abrupt-onset near the target induced a transient decrease in goal-directed discharge of SC visuomotor neurons. Although this was indicative of spatial competition, it was immediately followed by a rebound in presaccadic activation, which facilitated the saccadic response (i.e., it induced shorter saccadic RT). A similar suppression also occurred at most nontarget locations even in the absence of the abrupt-onset. This is indicative of a mechanism that enabled monkeys to quickly discount stimuli that shared the common nontarget feature. These results reveal a pattern of excitation/inhibition across the SC visuomotor map that acted to facilitate optimal behavior—the short duration suppression minimized the probability of capture by salient distractors, whereas a subsequent boost in accumulation rate ensured a fast goal-directed response. Such nonlinear dynamics should be incorporated into future biologically plausible models of saccade behavior.

During natural viewing, the trajectories of saccadic eye movements often deviate dramatically from a straight-line path between objects. In human studies, saccades have been shown to deviate toward or away from salient visual distractors depending on visual- and goal-related parameters, but the neurophysiological basis for this is not well understood. Some studies suggest that deviation toward is associated with competition between simultaneously active sites within the intermediate layers of the superior colliculus (SC), a midbrain structure that integrates sensory and goal-related signals for the production of saccades. In contrast, deviation away is hypothesized to reflect a higher-level process, whereby the neural site associated with the distractor isactively suppressed via a form of endogenous, top–down inhibition. We tested this hypothesis by measuring presaccadic distractor-evoked activation of SC visuomotor neurons while monkeys performed a simple task configured specifically to induce a high degree of saccades that deviate away. In the SC, cognitive processes such as top–down expectation are represented as variation in the sustained, low-frequency presaccadic discharge. We reasoned that any inhibition at the distractor-related locus associated with saccade deviation should affect the excitability of the neuron, thereby affecting the discharge rate. We found that, although the task produced robust deviation away, there was no evidence of a relationship between saccade deviation and distractor-evoked activation outside a short perisaccadic window that began no earlier than 22 msec before saccade onset. This indicates that deviation away is not adequately explained by a form of sustained, top–down inhibition at the distractor-related locus in the SC. The results are discussed in relation to the primary sources of inhibition associated with saccadic control.