DualKanbaFormer: Kolmogorov-Arnold Networks and State Space Model DualKanbaFormer: Kolmogorov-Arnold Networks and State Space Model Transformer for Multimodal Aspect-based Sentiment Analysis
Authors:
- Adamu LawanALAdamu Lawan
- Juhua PuJPJuhua Pu
- Haruna YunusaHYHaruna Yunusa
- Muhammad LawanMLMuhammad Lawan
- Aliyu UmarAUAliyu Umar
- Adamu Sani YahyaAYAdamu Sani Yahya
Preprint
Abstract
Multimodal aspect-based sentiment analysis (MABSA) enhances sentiment detection by combining text with other data types like images. However, despite setting significant benchmarks, attention mechanisms exhibit limitations in efficiently modelling long-range dependencies between aspect and opinion targets within the text. They also face challenges in capturing global-context dependencies for visual representations. To this end, we propose Kolmogorov-Arnold Networks (KANs) and Selective State Space model (Mamba) transformer (DualKanbaFormer), a novel architecture to address the above issues. We leverage the power of Mamba to capture global context dependencies, Multi-head Attention (MHA) to capture local context dependencies, and KANs to capture non-linear modelling patterns for both textual representations (textual KanbaFormer) and visual representations (visual KanbaFormer). Furthermore, we fuse the textual KanbaFormer and visual KanbaFomer with a gated fusion layer to capture the inter-modality dynamics. According to extensive experimental results, our model outperforms some state-of-the-art (SOTA) studies on two public datasets.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- AMIFN: Aspect-guided multi-view interactions and fusion network for multimodal aspect-based sentiment analysisNeurocomputing, 2024
- Cross-modal fine-grained alignment and fusion network for multimodal aspect-based sentiment analysisInformation Processing & Management, 2023
- Hierarchical Interactive Multimodal Transformer for Aspect-Based Multimodal Sentiment AnalysisIEEE Transactions on Affective Computing, 2022
- Multi-level Attention Map Network for Multimodal Sentiment AnalysisIEEE Transactions on Knowledge and Data Engineering, 2022
- Entity-Sensitive Attention and Fusion Network for Entity-Level Multimodal Sentiment ClassificationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019
- Deep Residual Learning for Image Recognition2016