Jiawen Deng
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Jiawen Deng has a total of 37 co-authors. Here are the top co-authors by number of shared publications:
- Fuji Ren
University of Electronic Science and Technology of China, Chengdu, Sichuan, China
12 shared publications - Yan Zhuang
University of Electronic Science and Technology of China, Chengdu, China
6 shared publications Fuji Ren
5 shared publications- Jiali You
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China
3 shared publications Satoshi Nakagawa
Graduate School of Information Science and Technology, University of Tokyo, Tokyo, Japan
3 shared publications- Yanru Zhang
University of Electronic Science and Technology of China
3 shared publications
Latest Publications
- 24 August 2026
- Published by Elsevier in Information Fusion
- Vol. 138
Frequency-aware alignment and modality-aware fusion for multimodal sentiment analysis
- 24 August 2026
- Published by Elsevier in Information Fusion
- Vol. 138
Frequency-aware alignment and modality-aware fusion for multimodal sentiment analysis
- 2 July 2026
- Published by Elsevier in Information Fusion
- Vol. 137
Salience-guided counterfactual framework for multimodal sentiment analysis
- 2 July 2026
- Published by Elsevier in Information Fusion
- Vol. 137
Salience-guided counterfactual framework for multimodal sentiment analysis
- Open Access
- 15 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1355-1364
ReNoRD: Learning from Relations under Noisy Pseudo Labels via Relational Distillation for Multimodal Sentiment
- 15 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1355-1364
Open AccessReNoRD: Learning from Relations under Noisy Pseudo Labels via Relational Distillation for Multimodal Sentiment
- Open Access
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-24
Breakdowns in Conversational AI: Interactional Failures in Emotionally and Ethically Sensitive Contexts
Conversational AI is increasingly deployed in emotionally charged and ethically sensitive interactions. Previous research has primarily concentrated on emotional benchmarks or static safety checks, overlooking how alignment unfolds in evolving conversation. We explore the research question: what breakdowns arise when conversational agents confront emotionally and ethically sensitive behaviors, and how do these affect dialogue quality? To stress-test chatbot performance, we develop a persona-conditioned user simulator capable of engaging in multi-turn dialogue with psychological personas and staged emotional pacing. Our analysis reveals that mainstream models exhibit recurrent breakdowns that intensify as emotional trajectories escalate. We identify several common failure patterns, including affective misalignments, ethical guidance failures, and cross-dimensional trade-offs where empathy supersedes or undermines responsibility. We organize these patterns into a taxonomy and discuss the design implications, highlighting the necessity to maintain ethical coherence and affective sensitivity throughout dynamic interactions. The study offers the HCI community a new perspective on the diagnosis and improvement of conversational AI in value-sensitive and emotionally charged contexts.
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-24
Open AccessBreakdowns in Conversational AI: Interactional Failures in Emotionally and Ethically Sensitive Contexts
Conversational AI is increasingly deployed in emotionally charged and ethically sensitive interactions. Previous research has primarily concentrated on emotional benchmarks or static safety checks, overlooking how alignment unfolds in evolving conversation. We explore the research question: what breakdowns arise when conversational agents confront emotionally and ethically sensitive behaviors, and how do these affect dialogue quality? To stress-test chatbot performance, we develop a persona-conditioned user simulator capable of engaging in multi-turn dialogue with psychological personas and staged emotional pacing. Our analysis reveals that mainstream models exhibit recurrent breakdowns that intensify as emotional trajectories escalate. We identify several common failure patterns, including affective misalignments, ethical guidance failures, and cross-dimensional trade-offs where empathy supersedes or undermines responsibility. We organize these patterns into a taxonomy and discuss the design implications, highlighting the necessity to maintain ethical coherence and affective sensitivity throughout dynamic interactions. The study offers the HCI community a new perspective on the diagnosis and improvement of conversational AI in value-sensitive and emotionally charged contexts.
- 27 February 2026
- Published by Elsevier in Neurocomputing
- Vol. 679
Decoupled hypergraph modeling for multimodal sentiment analysis
- 27 February 2026
- Published by Elsevier in Neurocomputing
- Vol. 679
Decoupled hypergraph modeling for multimodal sentiment analysis
- 18 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 1847-1859
Intra-Sample and Intra-Modal Enhancement for Multimodal Sentiment Analysis With Missing Modalities
Multimodal sentiment analysis (MSA) with missing modalities involves understanding the person's sentiment using multimodal data where some modalities are missing. Most existing methods focus on reconstructing the missing modalities using the available modalities from each sample, relying on modality-common information. However, these methods overlook the modality-specific information that other samples can provide. Additionally, these approaches often require the guidance of full modality representations during the reconstruction process, which is impractical in resource-constrained real-world scenarios. To address these challenges, we propose the Intra-sample and Intra-modal Enhancement (IIE) framework. The IIE framework enhances both sample-level and modality-level representations to capture additional modality-common and modality-specific information from existing modalities, without requiring full modalities. Specifically, IIE first learns sample-level representations by distilling modality-common information from the available modalities into learnable latent units. Then, it enhances modality-level representations by leveraging modality-specific information from other samples with the same modality, which is crucial for improving robustness in the presence of missing modalities. Finally, IIE ensures consistency between the enhanced modality-level and sample-level representations, combining the enhanced and initial representations to make predictions. Extensive experiments on three datasets demonstrate that the IIE framework significantly outperforms existing methods in terms of both effectiveness and robustness in handling MSA with missing modalities. Code is available at https://github.com/YetZzzzzz/IIE.
- 18 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 1847-1859
Intra-Sample and Intra-Modal Enhancement for Multimodal Sentiment Analysis With Missing Modalities
Multimodal sentiment analysis (MSA) with missing modalities involves understanding the person's sentiment using multimodal data where some modalities are missing. Most existing methods focus on reconstructing the missing modalities using the available modalities from each sample, relying on modality-common information. However, these methods overlook the modality-specific information that other samples can provide. Additionally, these approaches often require the guidance of full modality representations during the reconstruction process, which is impractical in resource-constrained real-world scenarios. To address these challenges, we propose the Intra-sample and Intra-modal Enhancement (IIE) framework. The IIE framework enhances both sample-level and modality-level representations to capture additional modality-common and modality-specific information from existing modalities, without requiring full modalities. Specifically, IIE first learns sample-level representations by distilling modality-common information from the available modalities into learnable latent units. Then, it enhances modality-level representations by leveraging modality-specific information from other samples with the same modality, which is crucial for improving robustness in the presence of missing modalities. Finally, IIE ensures consistency between the enhanced modality-level and sample-level representations, combining the enhanced and initial representations to make predictions. Extensive experiments on three datasets demonstrate that the IIE framework significantly outperforms existing methods in terms of both effectiveness and robustness in handling MSA with missing modalities. Code is available at https://github.com/YetZzzzzz/IIE.
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 14066-14072
FAME: Fusion-Aware Multi-modal Ensemble for Social Media Popularity Prediction
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 14066-14072
Open AccessFAME: Fusion-Aware Multi-modal Ensemble for Social Media Popularity Prediction
- 21 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 16 (4) , 3329-3341
Enhanced Emotion Recognition in Conversations Through Hybrid Context Encoding and Latent Dependency Mining
Emotion recognition in conversations (ERC) is a pivotal component of affective computing, involving a common two-stage paradigm where pre-trained language models first extract context-independent features, followed by the encoding of contextual information and the modeling of emotional dependencies. This paradigm faces two challenges: (1) Existing methods struggle to capture both the intra-dialogue emotional continuity and the inter-dialogue semantic similarity. (2) The complexity of emotional elicitation processes gives rise to entangled dependencies, termed “latent dependencies”, which are difficult for current methods to detect and analyze. To overcome these challenges, we propose a Hybrid-Context Encoder with an Automated Latent Dependency Mining model for ERC. Specifically, we examine the emotional continuity and the semantic similarity from the standpoint of context encoders. We experimentally find that context encoders with different architectures exhibit distinct benefits. Based on these findings, we design a hybrid contextual encoding module that effectively combines the strengths of various encoders. Additionally, we design a lightweight generative module for latent dependency mining that autonomously generates a context mask, enabling the effective discovery of latent dependencies. We conduct extensive experiments on three datasets in the text modality. Our model achieves the best performance, which validates the superiority of our approach.
- 21 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 16 (4) , 3329-3341
Enhanced Emotion Recognition in Conversations Through Hybrid Context Encoding and Latent Dependency Mining
Emotion recognition in conversations (ERC) is a pivotal component of affective computing, involving a common two-stage paradigm where pre-trained language models first extract context-independent features, followed by the encoding of contextual information and the modeling of emotional dependencies. This paradigm faces two challenges: (1) Existing methods struggle to capture both the intra-dialogue emotional continuity and the inter-dialogue semantic similarity. (2) The complexity of emotional elicitation processes gives rise to entangled dependencies, termed “latent dependencies”, which are difficult for current methods to detect and analyze. To overcome these challenges, we propose a Hybrid-Context Encoder with an Automated Latent Dependency Mining model for ERC. Specifically, we examine the emotional continuity and the semantic similarity from the standpoint of context encoders. We experimentally find that context encoders with different architectures exhibit distinct benefits. Based on these findings, we design a hybrid contextual encoding module that effectively combines the strengths of various encoders. Additionally, we design a lightweight generative module for latent dependency mining that autonomously generates a context mask, enabling the effective discovery of latent dependencies. We conduct extensive experiments on three datasets in the text modality. Our model achieves the best performance, which validates the superiority of our approach.
- 15 July 2025
- Published by Elsevier in Neurocomputing
- Vol. 651
Hierarchical Reasoning Enhanced Few-Shot Multimodal Sentiment Analysis
- 15 July 2025
- Published by Elsevier in Neurocomputing
- Vol. 651
Hierarchical Reasoning Enhanced Few-Shot Multimodal Sentiment Analysis
- Open Access
- 22 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 4160-4170
ETS-MM: A Multi-Modal Social Bot Detection Model Based on Enhanced Textual Semantic Representation
- 22 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 4160-4170
Open AccessETS-MM: A Multi-Modal Social Bot Detection Model Based on Enhanced Textual Semantic Representation