STSean Trott

Sean Trott

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Sean Trott has a total of 37 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Individual differences in visual imagery vividness predict neural responses to language-induced visual expectations

    by Ana Chkhaidze, Lera Boroditsky, Sean Trott, Anastasia Kiyonaga, Seana Coulson

    Grounded accounts of meaning propose that language comprehension recruits sensorimotor systems to represent described content. However, people differ markedly in how vividly they can picture objects and scenes in the mind's eye, raising the question of whether perceptual simulation is recruited similarly across individuals. We addressed this question using the sentence-picture verification task, in which pictures are verified more quickly when their shape matches rather than mismatches the state implied by a preceding sentence, a match effect taken to index perceptual simulation. In preregistered Experiment 1 (N = 110), matching pictures were verified faster and more accurately than mismatching pictures, yet the magnitude of this effect did not vary with imagery vividness. We reasoned that similar behavior might arise from different processes, so in Experiment 2 we recorded picture-locked event-related potentials (ERPs) from 34 participants performing the verification task. Shape mismatch elicited an early right-posterior response at 150-190 ms and a later, broader posterior response at 300-500 ms. Crucially, imagery vividness interacted with the shape match effect in the early window only: stronger imagers showed a pronounced right-posterior negativity, whereas verbal thinking tendency did not, indicating specificity to visual imagery. Together, results suggest that comparable behavioral outcomes may arise through partly different paths to meaning: an early, visually specific route recruited more strongly by vivid imagers, and a later conceptual route shared more broadly across the imagery spectrum. Language comprehension may draw on both perceptual simulation and linguistic associations, with the relative contribution of each depending in part on the comprehender.

  • Open Access

    Contextualized sensorimotor norms: Multi-dimensional measures of sensorimotor strength for ambiguous English words, in context

    by Sean Trott, Benjamin Bergen

    Embodied theories of language emphasize the role of sensorimotor experience in linguistic knowledge. Central to testing these theories is the creation of large datasets of linguistic norms , which contain judgments about a word’s sensorimotor associations and can be used to predict human behavioral or brain data – sometimes in contrast to competing variables, such as those derived from distributional language models. Yet many of these datasets contain judgments about words in isolation, despite the fact that most words are ambiguous, making it difficult to determine which meaning of a word is characterized by its rating (e.g., “wooden table” vs. “data table”). In the current work, we introduce a new lexical resource (directly inspired by the Lancaster sensorimotor for 112 English words, each rated in four different contexts (448 sentences total). We demonstrate: first, that these ratings encode overlapping but distinct information from the Lancaster sensorimotor norms; second, that decontextualized ratings likely reflect the more dominant meaning of ambiguous words; third, that homonyms have more distinct sensorimotor profiles than polysemes; fourth, that the contextualized sensorimotor distance between two uses of an ambiguous word predicts human judgments about semantic relatedness; and fifth, that ratings derived from GPT-4 align reasonably well with human judgments. We conclude by suggesting that contextualized ratings like these can be used both to inform competing theories of semantic representations and also to evaluate or “probe” the ability of LMs to recover sensorimotor information.


  • Open Access

    Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs

    by Sean Trott, Samuel Taylor, Cameron Jones, James A. Michaelov, Pamela D. Rivière

    Research on mental state reasoning in language models (LMs) has the potential to inform theories of human social cognition--such as the theory that mental state reasoning emerges in part from language exposure--and our understanding of LMs themselves. Yet much published work on LMs relies on a relatively small sample of closed-source LMs, limiting our ability to rigorously test psychological theories and evaluate LM capacities. Here, we replicate and extend published work on the false belief task by assessing LM mental state reasoning behavior across 41 open-weight models (from distinct model families). We find sensitivity to implied knowledge states in 34% of the LMs tested; however, consistent with prior work, none fully ``explain away'' the effect in humans. Larger LMs show increased sensitivity and also exhibit higher psychometric predictive power. Finally, we use LM behavior to generate and test a novel hypothesis about human cognition: both humans and LMs show a bias towards attributing false beliefs when knowledge states are cued using a non-factive verb (``John thinks...'') than when cued indirectly (``John looks in the...''). Unlike the primary effect of knowledge states, where human sensitivity exceeds that of LMs, the magnitude of the human knowledge cue effect falls squarely within the distribution of LM effect sizes-suggesting that distributional statistics of language can in principle account for the latter but not the former in humans. These results demonstrate the value of using larger samples of open-weight LMs to test theories of human cognition and evaluate LM capacities.

  • Open Access

    Capacity Constraints and the Multilingual Penalty for Lexical Disambiguation

    by Sean Trott, Pamela D. Rivière

    Multilingual language models (LMs) sometimes under-perform their monolingual counterparts, possibly due to capacity limitations. We quantify this ``multilingual penalty'' for lexical disambiguation--a task requiring precise semantic representations and contextualization mechanisms--using controlled datasets of human relatedness judgments for ambiguous words in both English and Spanish. Comparing monolingual and multilingual LMs from the same families, we find consistently reduced performance in multilingual LMs. We then explore three potential capacity constraints: representational (reduced embedding isotropy), attentional (reduced attention to disambiguating cues), and vocabulary-related (increased multi-token segmentation). Multilingual LMs show some evidence of all three limitations; moreover, these factors statistically account for the variance formerly attributed to a model's multilingual status. These findings suggest both that multilingual LMs do suffer from multiple capacity constraints, and that these constraints correlate with reduced disambiguation performance.

  • Open Access

    Start Making Sense(s): A Developmental Probe of Attention Specialization Using Lexical Ambiguity

    by Pamela D. Rivière, Sean Trott

    Despite an in-principle understanding of self-attention matrix operations in Transformer language models (LMs), it remains unclear precisely how these operations map onto interpretable computations or functions--and how or when individual attention heads develop specialized attention patterns. Here, we present a pipeline to systematically probe attention mechanisms, and we illustrate its value by leveraging lexical ambiguity--where a single word has multiple meanings--to isolate attention mechanisms that contribute to word sense disambiguation. We take a "developmental" approach: first, using publicly available Pythia LM checkpoints, we identify inflection points in disambiguation performance for each LM in the suite; in 14M and 410M, we identify heads whose attention to disambiguating words covaries with overall disambiguation performance across development. We then stress-test the robustness of these heads to stimulus perturbations: in 14M, we find limited robustness, but in 410M, we identify multiple heads with surprisingly generalizable behavior. Then, in a causal analysis, we find that ablating the target heads demonstrably impairs disambiguation performance, particularly in 14M. We additionally reproduce developmental analyses of 14M across all of its random seeds. Together, these results suggest: that disambiguation benefits from a constellation of mechanisms, some of which (especially in 14M) are highly sensitive to the position and part-of-speech of the disambiguating cue; and that larger models (410M) may contain heads with more robust disambiguation behavior. They also join a growing body of work that highlights the value of adopting a developmental perspective when probing LM mechanisms.


  • Open Access

    Toward a Theory of Generalizability in LLM Mechanistic Interpretability Research

    by Sean Trott

    Research on Large Language Models (LLMs) increasingly focuses on identifying mechanistic explanations for their behaviors, yet the field lacks clear principles for determining when (and how) findings from one model instance generalize to another. This paper addresses a fundamental epistemological challenge: given a mechanistic claim about a particular model, what justifies extrapolating this finding to other LLMs -- and along which dimensions might such generalizations hold? I propose five potential axes of correspondence along which mechanistic claims might generalize, including: functional (whether they satisfy the same functional criteria), developmental (whether they develop at similar points during pretraining), positional (whether they occupy similar absolute or relative positions), relational (whether they interact with other model components in similar ways), and configurational (whether they correspond to particular regions or structures in weight-space). To empirically validate this framework, I analyze "1-back attention heads" (components attending to previous tokens) across pretraining in random seeds of the Pythia models (14M, 70M, 160M, 410M). The results reveal striking consistency in the developmental trajectories of 1-back attention across models, while positional consistency is more limited. Moreover, seeds of larger models systematically show earlier onsets, steeper slopes, and higher peaks of 1-back attention. I also address possible objections to the arguments and proposals outlined here. Finally, I conclude by arguing that progress on the generalizability of mechanistic interpretability research will consist in mapping constitutive design properties of LLMs to their emergent behaviors and mechanisms.


  • Open Access

    Seeing Through Words, Speaking Through Pixels: Deep Representational Alignment Between Vision and Language Models

    by Zoe Wanying He, Sean Trott, Meenakshi Khosla

    Recent studies show that deep vision-only and language-only models--trained on disjoint modalities--nonetheless project their inputs into a partially aligned representational space. Yet we still lack a clear picture of where in each network this convergence emerges, what visual or linguistic cues support it, whether it captures human preferences in many-to-many image-text scenarios, and how aggregating exemplars of the same concept affects alignment. Here, we systematically investigate these questions. We find that alignment peaks in mid-to-late layers of both model types, reflecting a shift from modality-specific to conceptually shared representations. This alignment is robust to appearance-only changes but collapses when semantics are altered (e.g., object removal or word-order scrambling), highlighting that the shared code is truly semantic. Moving beyond the one-to-one image-caption paradigm, a forced-choice "Pick-a-Pic" task shows that human preferences for image-caption matches are mirrored in the embedding spaces across all vision-language model pairs. This pattern holds bidirectionally when multiple captions correspond to a single image, demonstrating that models capture fine-grained semantic distinctions akin to human judgments. Surprisingly, averaging embeddings across exemplars amplifies alignment rather than blurring detail. Together, our results demonstrate that unimodal networks converge on a shared semantic code that aligns with human judgments and strengthens with exemplar aggregation.

  • Open Access

    Turing Jest: Distributional Semantics and One‐Line Jokes

    by Sean Trott, Drew E. Walker, Samuel M. Taylor, Seana Coulson

    Humor is an essential aspect of human experience, yet surprisingly, little is known about how we recognize and understand humorous utterances. Most theories of humor emphasize the role of incongruity detection and resolution (e.g., frame‐shifting), as well as cognitive capacities like Theory of Mind and pragmatic reasoning. In multiple preregistered experiments, we ask whether and to what extent exposure to purely linguistic input can account for the human ability to recognize one‐line jokes and identify their entailments. We find that GPT‐3, a large language model (LLM) trained on only language data, exhibits above‐chance performance in tasks designed to test its ability to detect, appreciate, and comprehend jokes. In exploratory work, we also find above‐chance performance in humor detection and comprehension in several open‐source LLMs, such as Llama‐3 and Mixtral. Although all LLMs tested fall short of human performance, both humans and LLMs show a tendency to misclassify nonjokes with surprising endings as jokes. Results suggest that LLMs are remarkably adept at some tasks involving one‐line jokes, but reveal key limitations of distributional approaches to meaning.


  • Top Cited Papers
    Open Access

    AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracy

    by Philipp Schoenegger, Peter S. Park, Ezra Karger, Sean Trott, Philip E. Tetlock

    Large language models (LLMs) match and sometimes exceed human performance in many domains. This study explores the potential of LLMs to augment human judgment in a forecasting task. We evaluate the effect on human forecasters of two LLM assistants: one designed to provide high-quality (“superforecasting”) advice, and the other designed to be overconfident and base-rate neglecting, thus providing noisy forecasting advice. We compare participants using these assistants to a control group that received a less advanced model that did not provide numerical predictions or engage in explicit discussion of predictions. Participants ( N \(=\) 991) answered a set of six forecasting questions and had the option to consult their assigned LLM assistant throughout. Our preregistered analyses show that interacting with each of our frontier LLM assistants significantly enhances prediction accuracy by between 24% and 28% compared to the control group. Exploratory analyses showed a pronounced outlier effect in one forecasting item, without which we find that the superforecasting assistant increased accuracy by 41%, compared with 29% for the noisy assistant. We further examine whether LLM forecasting augmentation disproportionately benefits less skilled forecasters, degrades the wisdom-of-the-crowd by reducing prediction diversity, or varies in effectiveness with question difficulty. Our data do not consistently support these hypotheses. Our results suggest that access to a frontier LLM assistant, even a noisy one, can be a helpful decision aid in cognitively demanding tasks compared to a less powerful model that does not provide specific forecasting advice. However, the effects of outliers suggest that further research into the robustness of this pattern is needed.


  • Open Access

    Seeing Through Words, Speaking Through Pixels: Deep Representational Alignment Between Vision and Language Models

    by Zoe Wanying He, Sean Trott, Meenakshi Khosla

    Zoe Wanying He, Sean Trott, Meenakshi Khosla. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.