Multimodal AI approach to platform hate
Open Access
- 27 October 2026
- journal article
- Published by SAGE Publications in Discourse Studies
Abstract
A multimodal AI approach for detecting hate speech in social media and online platforms is explored in this study. It integrates AI techniques (Natural Language Processing (NLP), image processing, video processing and moderation models) to analyse multimodal communications across media platforms. The approach is illustrated using an interactive digital platform developed for analysing communications during the COVID-19 pandemic. The platform supports (a) collecting multimedia content from online sources; (b) processing text, images and videos; and (c) visualising results through an interactive dashboard. The findings indicate that publicly available AI tools for hate speech detection, in this case OpenAI moderation models (text-moderation 2023 and omni-moderation 2026) applied to text, OCR-extracted image text and transcribed video audio, are unreliable for identifying hate speech in real-life data. It is proposed that multimodal computational approaches informed by social semiotic theory would improve methods for automated hate speech detection.Keywords
Funding Information
- University of Liverpool
This publication has 17 references indexed in Scilit:
- Multimodal fusion techniques: Review, data representation, information fusion, and application areasNeurocomputing, 2025
- The DenseCap-Guided Attention Network For Image-Text Matching2025
- A transformer based multi task learning approach to multimodal hate speech detectionNatural Language Processing Journal, 2025
- Hate speech detection: A comprehensive review of recent worksExpert Systems, 2024
- A systematic review of hate speech automatic detection using natural language processingNeurocomputing, 2023
- Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate2021
- Algorithmic content moderation: Technical and political challenges in the automation of platform governanceBig Data & Society, 2020
- SFL and Multimodal Discourse Analysis2019
- War, Cinema, and Moral AnxietyAlternatives: Global, Local, Political, 2003
- Speech, Music, Sound1999