The bewitching AI: The Illusion of Communication with Large Language Models
Open Access
- 5 May 2025
- journal article
- research article
- Published by Springer Nature in Philosophy & Technology
- Vol. 38 (2) , 1-33
Abstract
We investigate the ‘bewitchment’ of understanding interactions between humans and systems based on large language models (LLMs) inspired by Wittgenstein’s later view on language. This framework is particularly apt for analyzing human-LLM interaction as it treats understanding as a public phenomenon manifested in observable communicative practices, rather than as a mental or computational state—an approach especially valuable given LLMs’ inherent opacity. Drawing on this perspective, we show that successful communication requires not merely regularity in language use, but constancy in maintaining reference points through agreement in both definitions and judgments. Crucially, LLMs lack the constancy needed to track negations and contradictions throughout a dialogue, thereby disrupting the reference points necessary for genuine communication. The apparent understanding in human-LLM interactions arises from what we characterize as a ‘bewitchment’: the interaction between LLMs’ statistical adherence to linguistic patterns and humans’ tendency to blindly follow familiar language games. Moreover, when interaction with LLMs is based on stereotyped contexts in which the system seems capable of identifying reference points, we humans automatically apply the practical principle that there is understanding until proven otherwise. The bewitchment becomes thus more profound as LLMs improve in modeling stereotypical aspects of human interaction. This improvement, far from addressing the highlighted limitations, can only deepen the illusion of understanding, raising significant concerns for meaningful control over such systems.Keywords
Funding Information
- Ministero dell'Università e della Ricerca (2020SSKZ7R, 2020SSKZ7R, 202223E8Y4X, PE 00000013, PE 00000013)
- Consiglio Nazionale delle Ricerche
This publication has 102 references indexed in Scilit:
- We are Building Gods: AI as the Anthropomorphised Authority of the PastMinds and Machines, 2024
- Revisiting the Turing Test: Humans, Machines, and Phraseology2023
- Machine understanding and deep learning representationSynthese, 2023
- Grounding the Vector Space of an Octopus: Word Meaning from Raw TextMinds and Machines, 2023
- Playing Games with Ais: The Limits of GPT-3 and Similar Large Language ModelsMinds and Machines, 2022
- Language and IntelligenceMinds and Machines, 2021
- GPT-3: Its Nature, Scope, Limits, and ConsequencesMinds and Machines, 2020
- The later Wittgenstein’s guide to contradictionsSynthese, 2019
- Negation on the Australian PlanJournal of Philosophical Logic, 2019
- The Autonomy of Grammar2016