Shiro Kumano

BackgroundWe proposed knowing sufficiency as a psychological construct comprising knowing what and how much is sufficient for one's well-being, not seeking beyond that point, and feeling content with what is already present in one's life. Existing psychological measures related to sufficiency primarily assess affective aspects such as dispositional contentment, but not the cognitive and behavioral aspects of recognizing what is enough and refraining from further pursuit. We therefore developed and validated the Knowing Sufficiency Scale (KSS) to assess individual differences across these cognitive, behavioral, and affective dimensions among Japanese adults.MethodsFollowing item generation and evaluation by experts and the target population, we collected two independent online survey samples, with final analytic sample sizes of n = 443 and n = 455. We conducted exploratory and confirmatory factor analyses and examined internal consistency, construct validity through correlations with related measures, and incremental validity. A further assessment was conducted approximately 2 weeks later, with 294 participants included in the test–retest analysis.ResultsThe resulting KSS comprised 19 items across three subscales: Knowing What Is Enough, Stopping at Enough, and Felt Sufficiency. A higher-order three-factor model showed acceptable fit on CFI, TLI, and SRMR, but not RMSEA, and fitted better than a one-factor model. Cronbach's α and McDonald's ω in both samples and test–retest ICCs were all at least .70, indicating acceptable to good reliability. The KSS overall and subscale scores generally showed positive associations with contentment, life satisfaction, savoring, gratitude, and psychological well-being, and negative associations with materialism, depressive symptoms, and ability-related social comparison orientation. Adding Knowing What Is Enough and Stopping at Enough to a model including age, gender, savoring, and gratitude increased the explained variance in contentment by 7.55 percentage points.ConclusionsThe findings provide initial evidence for the reliability and validity of the KSS. The scale offers a means of studying knowing sufficiency in terms of its cognitive, behavioral, and affective dimensions. Further research is needed to establish generalizability across languages and populations and to examine validity through associations with behavioral measures.

Empathy, a key element of social interaction, involves both cognitive and affective processes and is commonly investigated through measures such as empathic accuracy and affective physiological synchrony. While physiological synchrony offers a continuous measure of affective processes, empathic accuracy typically relies on discrete self-reports, leaving their temporal relationship largely unexplored. Advancing this line of research requires datasets that integrate time-continuous self-reports with physiological signals, yet such datasets—particularly those focusing on the empathizee—remain limited. To fill this gap, we present EMPAC (Empathy Measurement: Physiological, Affective, and Cognitive), a multimodal dataset constructed. To create empathy-eliciting stimuli, professional actors performed emotionally intense, pseudo-autobiographical narratives while their physiological signals (e.g., ECG, EDA) and continuous self-reported emotional states were recorded. We then conducted two observer experiments using these video recordings. In Experiment 1, to validate the stimuli as empathy-eliciting materials, observers continuously rated emotional intensity without being informed of the specific emotion portrayed, following the protocol of previous studies on time-series empathic accuracy. Yet this approach sometimes revealed a gap between the emotion category portrayed by the target and that perceived by the observers. In Experiment 2, we introduced a revised procedure in which the target emotion category was disclosed prior to viewing, revealing that specifying the target emotion led to a different relationship between individual empathy traits and empathic accuracy than observed in Experiment 1. EMPAC thus provides a rich, temporally aligned resource for investigating empathy dynamics in naturalistic settings and for evaluating methodological variations in empathic accuracy paradigms.

Humans align their physiology during social encounters, but the mechanisms driving this autonomic coupling remain unclear. Mimicry has been proposed as a driver, yet existing evidence is largely correlational and limited to precisely matched movements. We causally tested whether two forms of mimicry, copying a partner’s choices or motor actions, can induce heart rate synchrony during a controlled, non-verbal art selection task. In the Choice group (21 dyads), a confederate mimicked (or not) the participant’s picture choice but not their hand action. In the Motor group (20 dyads), a confederate mimicked (or not) the participant’s right-hand movements but chose from a different picture pair. Dyadic heart rate synchrony was quantified using mixed-effects models of trial-level heart rate fluctuations. Partners’ heart rates covaried within and across trials, but neither the presence nor the type of mimicry altered this coupling. After modelling temporal trends and comparing real dyads with time-matched pseudo-dyads, the observed synchrony was largely explained by shared task and timing dynamics rather than dyad-specific physiological alignment. Thus, instructed mimicry alone is insufficient to elicit dyad-specific cardiac coupling in minimal, tightly structured interactions. More broadly, these results caution against interpreting physiological synchrony as interpersonal alignment without accounting for shared temporal structure.

Humans align their physiology during social encounters, but the mechanisms driving this autonomic coupling remain unclear. Mimicry has been proposed as a driver, yet existing evidence is largely correlational and limited to precisely matched movements. We causally tested whether two forms of mimicry, copying a partner’s choices or motor actions, can induce heart rate synchrony during a controlled, non-verbal art selection task. In the Choice group (21 dyads), a confederate mimicked (or not) the participant’s picture choice but not their hand action. In the Motor group (20 dyads), a confederate mimicked (or not) the participant’s right-hand movements but chose from a different picture pair. Dyadic heart rate synchrony was quantified using mixed-effects models of trial-level heart rate fluctuations. Partners’ heart rates covaried within and across trials, but neither the presence nor the type of mimicry altered this coupling. After modelling temporal trends and comparing real dyads with time-matched pseudo-dyads, the observed synchrony was largely explained by shared task and timing dynamics rather than dyad-specific physiological alignment. Thus, instructed mimicry alone is insufficient to elicit dyad-specific cardiac coupling in minimal, tightly structured interactions. More broadly, these results caution against interpreting physiological synchrony as interpersonal alignment without accounting for shared temporal structure.

A novel framework named the synergistic functional spectrum analysis (sFSA) is proposed to explore the multifunctional interplay among multimodal nonverbal behaviours in human conversations. This study aims to reveal how multimodal nonverbal behaviours cooperatively perform communicative functions in conversations. To capture the intrinsic nature of nonverbal expressions, functional multiplicity, and interpretational ambiguity, e.g., a single head nod could imply listening, agreeing, or both, a novel concept named the functional spectrum , which is defined as the distribution of perceptual intensities of multiple functions by multiple observers, is introduced in the sFSA. Based on this concept, this paper presents functional spectrum corpora , which target 44 facial expression and 32 head movement functions. Then, spectrum decomposition is conducted to reduce the multimodal functional spectrum to a synergetic functional spectrum in a lower dimension functional space that is spanned by functional basis vectors representing primary and distinctive functionalities across multiple modalities. To that end, we propose a semiorthogonal nonnegative matrix factorization (SO-NMF) method, which assumes the additivity of multiple functions and aims to balance the distinctiveness and expressiveness of the factorization. The results confirm that some primary functional bases can be identified, which can be interpreted as the listener's backchannel, thinking, and affirmative response functions, and the speaker's thinking and addressing functions, and their positive emotion functions. In addition, regression models based on convolutional neural networks (CNNs) are presented to estimate the synergistic functional spectrum from the head poses and facial action units measured from conversation data. The results of these analyses and experiments confirm the potential of the sFSA and may lead to future extensions.

A novel framework named a gaze interactional functional spectrum analysis (GI-FSA) is proposed to explore the functional aspects of gaze interactions among interlocutors in conversations. It aims to reveal the primary and distinctive interactional functionalities that emerge via the gaze behaviors of the speaker, and the listener whom the speaker looks at. To capture the intrinsic nature of gaze functions, such as multiple functionalities and ambiguity, this study introduces a novel representation called a gaze functional spectrum representing the distribution of perceptual intensity of multiple gaze functions and presents a gaze functional spectrum corpus that targets 43 gaze functions covering various speech-related, listening-related and other functions. Then, semiorthogonal nonnegative matrix factorization (SO-NMF) is employed to decompose the concatenated speaker-listener functional spectra into a interactional functional spectrum in a lower-dimensional functional space spanned with functional bases, each of which represents a distinct aspect of interactional functionalities. Targeting four female conversations, the GI-FSA revealed interpretable functional bases such as addressing-listening and joint positive emotion. In addition, this paper proposes convolutional neural networks (CNNs) that can recognize the binary level of the interactional functional spectrum from observable multimodal nonverbal behaviors, including head pose, utterance status, eyeball direction and facial expressions. These experimental findings validate the potential of the GI-FSA as a promising framework for analyzing gaze interactions among interlocutors, and understanding communication dynamics.

The 9TH AAAC Conference on Affective Computing and Intelligent Interaction 2021 was held in a virtual format in the fall of 2021. It was technically co-sponsored by the IEEE Computer Society and featured the recent work on Affective Computing. The six best papers from this conference were selected by the technical program chairs. They were invited to submit their extended version to be considered for this special section at the IEEE Transactions on Affective Computing. Each submission was reviewed by at least three expert reviewers and was evaluated in terms of overall contribution and the adequacy of the additional content to warrant a new article. This special section features five accepted submissions whose major contributions are summarized below.

This paper introduces a novel approach to Item Response Theory (IRT) by incorporating deep learning to analyze student facial expressions to enhance the prediction and understanding of student responses to test items. This research is based on the assertion that students' facial expressions offer crucial insights into their cognitive and affective states during testing, subsequently influencing their item responses. The proposed State-Aware Deep Item Response Theory (SAD-IRT) model introduces a new parameter, the student state parameter, which can be viewed as a relative subjective difficulty parameter. It is latent-regressed from students' facial features while solving test items using state-of-the-art deep learning techniques. In an experiment with 20 students, SAD-IRT boosted prediction performance in students' responses compared to prior models without the student state parameter, including standard IRT and its deep neural network implementation, while maintaining consistent predictions of student ability and item difficulty parameters. The research further illustrates the model's early prediction ability in predicting the student's response result before the student answered. This study holds substantial implications for educational assessment, laying the groundwork for more personalized and effective learning and assessment strategies that consider students' emotional and cognitive states.