Jamie Reilly

Understanding speech requires listeners to integrate incoming input with prior linguistic and thematic knowledge to access meaning, a task greatly aided by prediction. Surprisal and related phenomena (e.g., next word prediction) tend to be associated with broad activation of language regions during listening. A major challenge for interpreting the neural correlates of surprisal is that lexical expectancies are constrained both by syntactic and semantic information. In contrast, semantic distance yields more of a pure metric of relatedness between one language constituent (e.g., word, phrase, n-gram) and another within a high-dimensional semantic space. We contrasted surprisal and semantic distance in running discourse during naturalistic audiovisual language comprehension. We analyzed fMRI scans from 20 English-speaking adults who viewed the full-length film 500 Days of Summer. Our focus was on brain sensitivity to either surprisal or semantic distance as language unfolded word-by-word. To this end, we implemented a series of hierarchical voxelwise encoding models. After accounting for low-level auditory and visual features, the addition of either surprisal or semantic distance improved prediction in bilateral superior temporal gyrus (STG), left inferior frontal gyrus, and right cerebellum. Critically, both semantic distance and surprisal explained unique variation in bilateral temporal cortex after accounting for the other. These results support the notion that the brain tracks both broad probabilistic prediction as well as semantic integration. More generally, these findings extend previous work on auditory-only speech by demonstrating that surprisal and semantic distance predict meaningful neural variation even within a rich audiovisual context.

Much of our scientific understanding of language processing has been informed by controlled experiments divorced from the real-world demands of naturalistic communication. Conversation requires synchronization of rate, amplitude, lexical complexity, affective coloring, shared reference, and countless other verbal and nonverbal dimensions. Conversation is not merely a vector for information transfer but also serves as a mechanism for establishing or maintaining social relationships. This process of language calibration between interlocutors is known as linguistic alignment . We developed an open-source R package, ConversationAlign , capable of computing novel indices of linguistic alignment and main effects of language use between interlocutors by evaluating word choice across numerous semantic, affective, and lexical dimensions (e.g., valence, concreteness, frequency, word length). We describe the operations of ConversationAlign, including its primary functions of cleaning and transforming raw language data into simultaneous time series objects aggregated by interlocutor, turn, and conversation. We then outline mathematical operations involved in computing complementary indices of linguistic alignment that capture both local (synchrony in turn-by-turn scores) and global relations (overall proximity) between interlocutors. We present a use case of ConversationAlign applied to interview transcripts from American radio legend Terry Gross and her many guests spanning 15 years. We identify caveats for use and potential sources of bias (e.g., polysemy, missing data, robustness to brief language samples) and close with a discussion of potential applications to other populations. ConversationAlign (v 0.4.0) is freely available for download and use via CRAN or GitHub. For technical instructions and download, visit https://github.com/Reilly-ConceptsCognitionLab/ConversationAlign .

Cleans and formats language transcripts guided by a series of transformation options (e.g., lemmatize words, omit stopwords, split strings across rows). 'SemanticDistance' computes two distinct metrics of cosine semantic distance (experiential and embedding). These values reflect pairwise cosine distance between different elements or chunks of a language sample. 'SemanticDistance' can process monologues (e.g., stories, ordered text), dialogues (e.g., conversation transcripts), word pairs arrayed in columns, and unordered word lists. Users specify options for how they wish to chunk distance calculations. These options include: rolling ngram-to-word distance (window of n-words to each new word), ngram-to-ngram distance (2-word chunk to the next 2-word chunk), pairwise distance between words arrayed in columns, matrix comparisons (i.e., all possible pairwise distances between words in an unordered list), turn-by-turn distance (talker to talker in a dialogue transcript). 'SemanticDistance' includes visualization options for analyzing distances as time series data and simple semantic network dynamics (e.g., clustering, undirected graph network).

A defining characteristic of social complexity in Homo sapiens is the diversity of our relationships. We build connections of various types in our families, workplaces, neighbourhoods and online communities. How do we make sense of such complex systems of human relationships? The basic organization of relationships has long been studied in the social sciences, but no consensus has been reached. Here, by using online surveys, laboratory cognitive tasks and natural language processing in diverse modern cultures across the world (n = 20,427) and ancient cultures spanning 3,000 years of history, we examined universality and cultural variability in the ways that people conceptualize relationships. We discovered a universal representational space for relationship concepts, comprising five principal dimensions (formality, activeness, valence, exchange and equality) and three core categories (hostile, public and private relationships). Our work reveals the fundamental cognitive constructs and cultural principles of human relationship knowledge and advances our understanding of human sociality.

Much of our understanding of language processing has been informed by controlled laboratory experiments abstracted from the real world demands of naturalistic communication. Interactive language use “in the wild” involves synchronizing numerous verbal and non-verbal behaviors between interlocutors. Conversation partners synchronize verbal production through modulations of rate, amplitude, lexical-semantic complexity, affective tone, and many other dimensions. Much remains to be learned about how to measure linguistic alignment in naturalistic interactions. We developed an open-source R package (ConversationAlign) capable of computing novel indices of alignment and main effects of language use between interlocutors across 30 psycholinguistic dimensions (e.g., valence, concreteness, frequency, word length). We describe operations of the ConversationAlign workflow, including its primary functions of cleaning transcripts and transforming raw language data to simultaneous time series objects aggregated by interlocutor, turn, and conversation. We present a use case of ConversationAlign applied to interview transcripts between American radio legend, Terry Gross, and her many guests over the span of 15 years (Fresh Air Archive, 2001). We identify caveats for use and potential sources of bias (e.g., polysemy, missing data, robustness to brief language samples). We close with a discussion of potential applications to better understand lexical alignment. ConversationAlign is freely available for download and use via GitHub at https://github.com/Reilly-ConceptsCognitionLab/ConversationAlign

Our earliest acquired words tend to reference concrete objects that can be seen, heard, touched, and felt. One hallmark of language maturation involves an ‘abstractness shift’ where language is used to convey intangible mental states, emotions, and ideas. Much of our understanding of the trajectory of concreteness in language development ends at early adolescence and is informed by studies of comprehension. We investigated concreteness effects in language production across the span of adulthood with a focus on conversation. We tested two competing hypotheses about abstract word use in conversation as a function of normal aging. The first hypothesis is that a steady accrual of vocabulary knowledge over the lifespan will confer a more expansive abstract lexicon and that older adults will in turn produce more abstract words in conversation. An alternative hypothesis is that aging gradually compromises executive resources that impact abstract word retrieval, leading to a concreteness bias in conversation. We analyzed distributions of abstract and concrete words and cross-speaker alignment within unscripted conversations between adults of different ages (N=1565 conversations, >8 million words, age range 19-66) using a novel computational algorithm (ConversationAlign). Aging was associated with abstract word dropout, a finding that is consistent with the hypothesis that high processing demands of conversation (e.g., alignment, topic maintenance) bias lexical retrieval toward concrete word use. We discuss the significance of these results for understanding intergenerational communication and relationships between aging, executive functioning, and lexical retrieval in high-level discourse.