Seeing Sarcasm Through Different Eyes: Analyzing Multimodal Sarcasm Perception in Large Vision-Language Models
- 17 October 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Social Systems
- Vol. 13 (2) , 2122-2139
Abstract
With the advent of large vision-language models (LVLMs) demonstrating increasingly human-like abilities, a pivotal question emerges: do different LVLMs interpret multimodal sarcasm differently, and can a single model grasp sarcasm from multiple perspectives like humans? To explore this, we introduce an analytical framework using systematically designed prompts on existing multimodal sarcasm datasets. Evaluating 12 state-of-the-art LVLMs over 2409 samples, we examine interpretive variations within and across models, focusing on confidence levels, alignment with dataset labels, and recognition of ambiguous “neutral” cases. We further validate our findings on a diverse 100-sample mini-benchmark, incorporating multiple datasets, expanded prompt variants, and representative commercial LVLMs. Our findings reveal notable discrepancies—across LVLMs and within the same model under varied prompts. While classification-oriented prompts yield higher internal consistency, models diverge markedly when tasked with interpretive reasoning. These results challenge binary labeling paradigms by highlighting sarcasm’s subjectivity. We advocate moving beyond rigid annotation schemes toward multiperspective, uncertainty-aware modeling, offering deeper insights into multimodal sarcasm comprehension.Keywords
All Related Versions
Funding Information
- Natural Science Foundation of Anhui Province (2308085MF220)
- Natural Science Foundation of Anhui Province (2023AH050914)
This publication has 32 references indexed in Scilit:
- SarcasmBench: Towards Evaluating Large Language Models on Sarcasm UnderstandingIEEE Transactions on Affective Computing, 2025
- RCLMuFN: Relational context learning and multiplex fusion network for multimodal sarcasm detectionKnowledge-Based Systems, 2025
- M2IST: Multi-Modal Interactive Side-Tuning for Efficient Referring Expression ComprehensionIEEE Transactions on Circuits and Systems for Video Technology, 2025
- How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suitesScience China Information Sciences, 2024
- SINCERE: A Hybrid Framework With Graph-Based Compact Textual Models Using Emotion Classification and Sentiment Analysis for Twitter Sarcasm DetectionIEEE Transactions on Computational Social Systems, 2024
- Improved Baselines with Visual Instruction Tuning2024
- Hybrid Quantum-Classical Neural Network for Multimodal Multitask Sarcasm, Emotion, and Sentiment AnalysisIEEE Transactions on Computational Social Systems, 2024
- An Emoticon-Based Novel Sarcasm Pattern Detection Strategy to Identify Sarcasm in Microblogging Social NetworksIEEE Transactions on Computational Social Systems, 2023
- DIP: Dual Incongruity Perceiving Network for Sarcasm Detection2023
- Sentiment analysis for sarcasm detection on streaming short text data2017