Tripartite Intelligence: Synergizing Deep Neural Network, Large Language Model, and Human Intelligence for Public Health Misinformation Detection (Archival Full Paper)
Open Access
- 27 June 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 63-75
Abstract
The threat of rapidly spreading health misinformation through social media during crises like COVID-19 emphasizes the importance of addressing both clear falsehoods and complex misinformation, including conspiracy theories and subtle distortions. This paper designs a novel tripartite collective intelligence approach that integrates deep neural networks (DNNs), large language models (LLMs), and crowdsourced human intelligence (HI) to collaboratively detect complex forms of public health misinformation on social media. Our design is inspired by the collaborative strengths of DNNs, LLMs, and HI, which complement each other. We observe that DNNs efficiently handle large datasets for initial misinformation screening but struggle with complex content and rely on high-quality training data. LLMs enhance misinformation detection with improved language understanding but may sometimes provide eloquent yet factually incorrect explanations, risking misinformation mislabeling. HI provides critical thinking and ethical judgment superior to DNNs and LLMs but is slower and more costly in misinformation detection. In particular, we develop TriIntel, a tripartite collaborative intelligence framework that leverages the collective intelligence of DNNs, LLMs, and HI to tackle the public health information detection problem under a novel few-shot and uncertainty-aware maximum likelihood estimation framework. Evaluation results on a real-world public health misinformation detection application related to COVID-19 show that TriIntel outperforms representative DNNs, LLMs, and human-AI collaboration baselines in accurately detecting public health misinformation under a diverse set of evaluation scenarios.Keywords
Funding Information
- National Science Foundation (IIS-2202481, CHE-2105032, IIS-2130263, CNS-2131622, CNS-2140999)
This publication has 41 references indexed in Scilit:
- Generative AI and ChatGPT: Applications, challenges, and AI-human collaborationJournal of Information Technology Case and Application Research, 2023
- Securing social platform from misinformation using deep learningComputer Standards & Interfaces, 2022
- A Comprehensive Review on Fake News Detection With Deep LearningIEEE Access, 2021
- A Mathematical Analysis of Learning Loss for Active Learning in Regression2021
- An Analysis On Sarcasm Detection Over Twitter During COVID-192021
- Leveraging multimodal social media data for rapid disaster damage assessmentInternational Journal of Disaster Risk Reduction, 2020
- Deep learning for misinformation detection on online social networks: a survey and new perspectivesSocial Network Analysis and Mining, 2020
- Task Effects on Linguistic Complexity and Accuracy: A Large‐Scale Learner Corpus Analysis Employing Natural Language Processing TechniquesLanguage Learning, 2017
- Collective IntelligenceBusiness & Information Systems Engineering, 2010
- Developing a collective intelligence application for special educationDecision Support Systems, 2009