Kirsten Ziman

Understanding someone else’s attention lies at the heart of human interaction. When we perceive something to be in someone else’s attention, we understand it to be in that person’s mind in the moment, ready to affect that person’s choices. It has been proposed that people construct predictive models of the attentional state of others. In that proposal, a working understanding of someone else’s attention goes far beyond merely tracking the direction of someone’s eyes. For example, in a recent study, participants watched a spotlight of attention moving around a picture and successfully judged whether the trace represented real, human attention or an artificially manipulated version, demonstrating implicit information about how patterns of attention behave over space and time. However, the neuronal underpinnings of attention modeling are almost entirely unstudied. Here we tested people in an fMRI scanner while they performed the attention judgment paradigm. When contrasting attention traces that participants judged to be artificial versus those judged to be real, activity was found bilaterally in the precuneus, dorsomedial prefrontal cortex, anterior cingulate, and anterior insula, as well as in a part of the right temporoparietal junction. These areas partially overlap the theory-of-mind network and the salience network. The activations differ from those known to be involved in processing low-level features of attention such as the sight of other people’s eyes. The results provide an initial picture of the cortical networks involved in monitoring the attention patterns of others and recognizing when those patterns deviate from expectation.

As social animals, people are highly sensitive to the attention of others. Seeing someone else gaze at an object automatically draws one’s own attention to that object. Monitoring the attention of others aids in reconstructing their emotions, beliefs, and intentions and may play a crucial role in social alignment. Recently, however, it has been suggested that the human brain constructs a predictive model of other people’s attention that is far more involved than a moment-by-moment monitoring of gaze direction. The hypothesized model learns the statistical patterns in other people’s attention and extrapolates how attention is likely to move. Here, we tested the hypothesis of a predictive model of attention. Subjects saw movies of attention displayed as a bright spot shifting around a scene. Subjects were able to correctly distinguish natural attention sequences (based on eye tracking of prior participants) from altered sequences (e.g., played backward or in a scrambled order). Even when the attention spot moved around a blank background, subjects could distinguish natural from scrambled sequences, suggesting a sensitivity to the spatial–temporal statistics of attention. Subjects also showed an ability to recognize the attention patterns of different individuals. These results suggest that people possess a sophisticated model of the normal statistics of attention and can identify deviations from the model. Monitoring attention is therefore more than simply registering where someone else’s eyes are pointing. It involves predictive modeling, which may contribute to our remarkable social ability to predict the mind states and behavior of others.

Our subjective sense that something we encounter is familiar to us is reflected by changes in pupil size. Although pupil dilation effects of familiarity have been well documented, familiarity is not the only, or even the strongest, contributing factor to pupil dilation. Changes in pupil dilation also reflect changes in brightness, affective or emotional responses, hormonal release, expected value or utility, and surprise, among others. Because many factors can affect pupil dilation, important questions remain about how pupil dilation changes reflect high-order cognitive processes, like attention and memory. For example, because surprise and familiarity are often difficult to fully distinguish (since new experiences can be surprising or unexpected), it can be difficult to tease apart pupil dilation effects of surprise versus familiarity. To better understand the effects of surprise and familiarity on pupil dilation, we examined pupil responses during a recognition memory task involving photographs of faces and outdoor scenes. When participants rated novel face images as “familiar,” we observed a robust pupil dilation response.

Our ongoing subjective experiences, and our memories of those experiences, are shaped by our prior experiences, goals, and situational understanding. These factors influence how we allocate our attentional resources over different aspects of our ongoing experiences. These attentional shifts may happen overtly (e.g., when we change where we are looking) or covertly (e.g., without any external physical manifestation). Additionally, we may attend to what is happening at a specific spatial location (e.g., because we think something important is happening there) or we may attend to particular features irrespective of their locations (e.g., when we search for a friend's face in a crowd versus a desired item in a grocery store). We ran a covert attention experiment with two conditions that asked participants to either sustain or vary the focus of the categories and locations they were attending. Later, the participants performed a recognition memory task for attended, unattended, and novel stimuli. Participants in both conditions recognized attended stimuli more readily than unattended or novel stimuli. Participants in the "sustained attention" condition also exhibited a recognition advantage for attended-category images from the unattended location, whereas participants in the "variable attention" condition exhibited a recognition advantage for attended-location images from the unattended category. Our findings suggest that covert attention enhances memory encoding of attended stimuli, and that different aspects of partial attention affect memory encoding over different timescales.

Verbal responses are a convenient and naturalistic way for participants to provide data in psychological experiments (Salzinger, The Journal of General Psychology, 61(1),65–94:1959). However, audio...

Quail is a Python package for analyzing and plotting behavioral data from memory experiments. The current focus is on free recall experiments (Kahana 2012; Manning, Norman, and Kahana 2015). Quail implements classic and more recently developed memory analyses and provides easy plotting functions by wrapping Seaborn (Waskom et al. 2016). API documentation, tutorials and examples can be found on our readthedocs page [link].

Verbal responses are a convenient and naturalistic way for participants to provide data in psychological experiments (Salzinger, 1959). However, audio recordings of verbal responses typically require additional processing such as transcribing the recordings into text, as compared with other behavioral response modalities (e.g. typed responses, button presses, etc.). Further, the transcription process is often tedious and time-intensive, requiring human listeners to manually examine each moment of recorded speech. Here we evaluate the performance of a state-of-the-art speech recognition algorithm (Halpern et al., 2016) in transcribing audio data into text during a list-learning experiment. We compare the computer-generated transcripts to transcripts made by human annotators. Both sets of transcripts matched to a high degree and exhibited similar statistical properties, in terms of the participants' recall performance and recall dynamics that the transcripts captured. This proof-of-concept study suggests that speech-to-text engines could provide a cheap, reliable, and rapid means of automatically transcribing speech data in psychological experiments. Further, our findings open the door for verbal response experiments that scale to thousands of participants (e.g. administered online), as well as a new generation of experiments that decode speech on-the-fly and adapt experimental parameters based on participants' prior responses.