The Illusion of Causality in LLMs: A Developmentally Grounded Analysis of Semantic Scaffolding and Benchmark–Capability Mismatches

Author:
Open Access
Publisher website
Google Scholar
Abstract
Recent benchmarks increasingly report that large language models (LLMs) exhibit human-like causal reasoning abilities, including counterfactual inference and intervention planning. However, many such evaluations rely on domains that are heavily represented in training data and embed strong semantic cues, raising the possibility that apparent causal competence may reflect semantic pattern recombination rather than structure-sensitive causal reasoning. Drawing on human developmental theories of causal induction, this perspective argues that genuine causal understanding requires robustness to novelty and reliance on conditional structure rather than semantic familiarity. To illustrate the testability of this claim, the paper includes a pilot demonstration using synthetic causal micro-worlds. Identical numerical evidence was presented to a LLM under two conditions: semantically meaningful variable labels and non-semantic coded labels. Across paired cases, the model reliably selected the correct causal structure when labels were meaningful, but frequently misidentified or exhibited instability in causal model selection under coded labels, despite producing locally coherent explanations. These divergences emerged most clearly in diagnostically ambiguous settings requiring suppression of misleading marginal associations. The results align with the claim that semantic scaffolding can support and stabilize apparent causal competence in LLMs without implying structure-sensitive reasoning.
Funding Information
  • Robin Hood Foundation (A1N-5A000000IYl2QAA)
  • City University of New York Central Office

This publication has 19 references indexed in Scilit: