Luke J. Chang

In this chapter, we outline important considerations for designing and analyzing conversation studies. We first advocate for the use of representative designs to ensure that datasets adequately reflect the complexity inherent in conversation. We then focus on how behavior in conversation recordings can be quantified through a process of feature extraction, and discuss strategies to assess the validity of feature extraction pipelines. Finally, we discuss analytical considerations specific to working with conversation data. Being able to rigorously collect, operationalize, and analyze natural conversation data will lead to robust results that the entire field can build upon, advancing our understanding of exactly how conversation allows us to achieve our social goals.

A crucial feature of human social cohesion is a feeling of responsibility when we cause harm to others, and an associated willingness to repair any damage we have caused. However, it is largely unknown to what degree these tendencies are evident when the harm we cause has been unintentional. Do we provide reparations to rectify the unintended consequences of the actions? In the current study we explore this question, focusing on the role of responsibility as a potential motivator of reparative behavior. In a novel, incentive-compatible, group decision-making task, we addressed these questions by leveraging both computational modeling and functional neuroimaging. In our sample (N = 48), we found that feelings of responsibility for inflicting unintended harm were computationally tractable: a group’s collective responsibility reflected the psychophysics of diminishing sensitivity, diffused solely over those who were accountable for inflicting the harm, and was agnostic to dissenting votes. We found evidence that these feelings of responsibility were encoded in several regions of the brain, including the orbitofrontal cortex, the posterior cingulate cortex, and the amygdala. In terms of reparative action, individual preferences for rectifying harm predicted decisions to subsequently repair harm, irrespective of whether the individual themselves caused it.

After just a few minutes of conversation, we may feel ``on the same page'', or profoundly disconnected, based on thin evidence.How do brief interactions produce such sweeping expectations about what we share with others?We propose that generalized shared reality arises through structured inductive inference: people leverage intuitive knowledge about how beliefs covary to generalize from sparse observations.Using a text-based chat paradigm, we asked pairs of strangers to discuss either shared or opposing beliefs, and compared these conversations with a control condition in which participants simply learned their partner's stance.We observed systematic generalization gradients: after observing agreement or disagreement on one topic, people expected more or less similarity on related topics, respectively. Real conversations relaxed thresholds for inferring commonality, amplifying these expectations.Participants who inferred broader commonalities reported stronger shared reality, linking these structured inferences to the subjective experience of connection.A Bayesian factor model that leverages population-level belief covariance captured participants' generalization patterns, whereas an egocentric heuristic model did not. Large language models given full conversation transcripts better matched participants' judgments about broader commonalities, confirming that conversations convey structured information beyond a single observed stance. Taken together, these findings illuminate how brief interactions can systematically shape our expectations of what we have in common with others through structured inductive inference over belief covariance.

Summary: Humans effortlessly learn attributes of other individuals and their complex web of connections through navigating the social world ^1,2 . Yet, the neural mechanisms that transform these transient interactions into structured, multidimensional knowledge remain unknown ^3,4 . Here, using a naturalistic fMRI paradigm ⁵ , we develop a computational framework to demonstrate how the human brain factorizes and integrates dynamic social interactions to construct multiplex social graphs. This approach not only predicts neural responses during movie-viewing but also allows for the reconstruction of subjective social cognitive maps directly from brain activity. Crucially, the relational geometry of these reconstructed maps accurately predicts inferred personality traits, indicating that relational and trait knowledge emerge from a shared neural representation reflecting interactional dynamics. These findings reveal an organizing computational principle by which the brain transforms dynamic social experiences into structured cognitive maps ⁶ , providing a key mechanism for the emergence of multiplex social knowledge in the human mind.

Close friends seem to communicate almost telepathically. A single word or phrase can be loaded with meaning that would take pages of text to explain to an outsider. This linguistic compression is undeniably efficient. But is efficiency the only reason friends often speak in code? In a series of studies, we show that when people use “insider language” with each other they feel closer. Friends, as expected, use insider language more frequently than strangers, but even strangers are able to incorporate it into their conversations. While the content of insider language varies between friends and strangers, both groups experience heightened connection when it is used. These results suggest that the human ability to say more with less confers a social advantage beyond mere efficiency. These findings also highlight a potential limitation of large language models, which, trained on explicit text, may struggle to grasp how communication shifts with intimacy and may be unable to generate insider language to foster closeness.

Inspired by inductive reasoning models, we test whether generalized shared reality (i.e., the sense of being on the same page) arises through probabilistic inference about latent commonalities. Using a naturalistic text-based chat paradigm, we manipulated whether conversation partners discussed a belief they shared, a belief on which their opinions differed, or a random prompt. Participants discussing shared opinions reported experiencing greater shared reality compared to those discussing differences or random topics. Moreover, participants who made broader inferences about additional beliefs they might share with their partners also reported greater shared reality. While discussing shared opinions can induce an overall greater sense of shared reality, participants discussing differences leveraged their conversation to establish shared realities about other topics. We demonstrate that shared reality can emerge in multiple ways during initial interactions, establishing a foundation for future mechanistic investigations within an inductive inference framework.

Receiving a favor from another person may induce a negative feeling of indebtedness for the beneficiary. In this study, we explore these hidden costs by developing and validating a conceptual model of indebtedness across three studies that combine a large-scale online questionnaire, an interpersonal game, computational modeling, and neuroimaging. Our model captures how individuals perceive the altruistic and strategic intentions of the benefactor. These inferences produce distinct feelings of guilt and obligation that together comprise indebtedness and motivate reciprocity. Perceived altruistic intentions convey care and communal concern and are associated with activity in insula, ventromedial prefrontal cortex and dorsolateral prefrontal cortex, while inferred strategic intentions convey expectations of future reciprocity and are associated with activation in temporal parietal junction and dorsomedial prefrontal cortex. We further develop a neural utility model of indebtedness using multivariate patterns of brain activity that captures the tradeoff between these feelings and reliably predicts reciprocity behavior.

People structure their days to experience events with others. We gather to eat meals, watch TV, and attend concerts together. What constitutes a shared experience and how does it manifest in dyadic behavior? The present study investigates how shared experiences—measured through emotional, motoric, physiological, and cognitive alignment—promote social bonding. We recorded the facial expressions and electrodermal activity (EDA) of participants as they watched four episodes of a TV show for a total of 4 h with another participant. Participants displayed temporally synchronized and spatially aligned emotional facial expressions and the degree of synchronization predicted the self-reported social connection ratings between viewing partners. We observed a similar pattern of results for dyadic physiological synchrony measured via EDA and their cognitive impressions of the characters. All four of these factors, temporal synchrony of positive facial expressions, spatial alignment of expressions, EDA synchrony, and character impression similarity, contributed to a latent factor of a shared experience that predicted social connection. Our findings suggest that the development of interpersonal affiliations in shared experiences emerges from shared affective experiences comprising synchronous processes and demonstrate that these complex interpersonal processes can be studied in a holistic and multi-modal framework leveraging naturalistic experimental designs.

Trust is a nebulous construct central to successful cooperative exchanges and interpersonal relationships. In this study, we introduce a new approach to establishing construct validity of trust using “neurometrics”. We develop a whole-brain multivariate pattern capable of classifying whether new participants will trust a relationship partner in the context of a cooperative interpersonal investment game (n=40) with 90% accuracy and find that it also generalizes to a variant of the same task collected in a different country with 82% accuracy (n=17). Moreover, we establish the convergent and discriminant validity by testing the pattern on thirteen separate datasets (n=1,427) and find that trust is reliably related to beliefs of safety, inversely related to negative affect, but unrelated to reward, cognitive control, social perception, theory of mind, and self-referential processing. Together these results provide support for the notion that the psychological experience of trust contains elements of beliefs of reciprocation and fear of betrayal aversion. Contrary to our predictions, we found no evidence that trust is related to anticipated reward. This work demonstrates how “neurometrics” can be used to characterize the psychological processes associated with brain-based multivariate representations.

Studying facial expressions is a notoriously difficult endeavor. Recent advances in the field of affective computing have yielded impressive progress in automatically detecting facial expressions from pictures and videos. However, much of this work has yet to be widely disseminated in social science domains such as psychology. Current state-of-the-art models require considerable domain expertise that is not traditionally incorporated into social science training programs. Furthermore, there is a notable absence of user-friendly and open-source software that provides a comprehensive set of tools and functions that support facial expression research. In this paper, we introduce Py-Feat, an open-source Python toolbox that provides support for detecting, preprocessing, analyzing, and visualizing facial expression data. Py-Feat makes it easy for domain experts to disseminate and benchmark computer vision models and also for end users to quickly process, analyze, and visualize face expression data. We hope this platform will facilitate increased use of facial expression data in human behavior research.