Menghua Jiang
Menghua Jiang has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Haifeng Hu
Sun Yat-Sen University, Guangzhou, China
1 shared publication - Leheng Zhang
South China Normal University, Guangzhou, Guangdong, China
1 shared publication - Manning Gao
South China Normal University, Guangzhou, China
1 shared publication Sijie Mai
1 shared publication- Yewen Huang
School of Electronics and Information Technology, Sun Yat-sen University, 510006
1 shared publication Yuxia Lin
1 shared publication
Latest Publications
- Open Access
- 14 September 2026
- Published by Elsevier in SSRN Electronic Journal
PRISM: Primary-Centric Multimodal Fusion with Information Bottlenecks and Confidence-Aware Gating
Multimodal sentiment analysis predicts continuous or ordinal affect from language, acoustic, and visual cues, but cross-modal alignment and per-stream reliability vary by sample. Previous fusion often collapses modalities into a symmetric or text-fixed representation, which may amplify inherent noise and ignore important information. Therefore, we propose PRISM, a primary-centric framework:(i) each modality is independently encoded by a parallel \emph{Variational Token Bottleneck} (\VTB) under a variational information-bottleneck objective, producing compact token-level latents together with token-level confidence estimates that quantify per-token reliability;(ii) a \emph{Differentiable Primary Router} (\DPR) then applies a Gumbel--Softmax relaxation to select a sample-dependent primary pathway from these bottleneck representations, rather than assuming a fixed dominant modality, and routes prediction exclusively through the enhanced primary stream instead of a pooled multimodal vector; and(iii) stacked InfoGate layers refine the selected primary by confidence-aware auxiliary-to-primary attention and adaptive auxiliary gating, using the \VTB confidence maps to downweight unreliable auxiliary tokens.We evaluate the model on both regression and classification datasets. While showing strong performance on regression tasks, the architecture also generalizes to binary humor and sarcasm classification.
- 14 September 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessPRISM: Primary-Centric Multimodal Fusion with Information Bottlenecks and Confidence-Aware Gating
Multimodal sentiment analysis predicts continuous or ordinal affect from language, acoustic, and visual cues, but cross-modal alignment and per-stream reliability vary by sample. Previous fusion often collapses modalities into a symmetric or text-fixed representation, which may amplify inherent noise and ignore important information. Therefore, we propose PRISM, a primary-centric framework:(i) each modality is independently encoded by a parallel \emph{Variational Token Bottleneck} (\VTB) under a variational information-bottleneck objective, producing compact token-level latents together with token-level confidence estimates that quantify per-token reliability;(ii) a \emph{Differentiable Primary Router} (\DPR) then applies a Gumbel--Softmax relaxation to select a sample-dependent primary pathway from these bottleneck representations, rather than assuming a fixed dominant modality, and routes prediction exclusively through the enhanced primary stream instead of a pooled multimodal vector; and(iii) stacked InfoGate layers refine the selected primary by confidence-aware auxiliary-to-primary attention and adaptive auxiliary gating, using the \VTB confidence maps to downweight unreliable auxiliary tokens.We evaluate the model on both regression and classification datasets. While showing strong performance on regression tasks, the architecture also generalizes to binary humor and sarcasm classification.