Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
Authors:
Preprint
Abstract
The impressive performance of recent language models across a wide range of tasks suggests that they possess a degree of abstract reasoning skills. Are these skills general and transferable, or specialized to specific tasks seen during pretraining? To disentangle these effects, we propose an evaluation framework based on "counterfactual" task variants that deviate from the default assumptions underlying standard tasks. Across a suite of 11 tasks, we observe nontrivial performance on the counterfactual variants, but nevertheless find that performance substantially and consistently degrades compared to the default conditions. This suggests that while current LMs may possess abstract task-solving skills to an extent, they often also rely on narrow, non-transferable procedures for task-solving. These results motivate a more careful interpretation of language model performance that teases apart these aspects of behavior.Keywords
All Related Versions
This publication has 39 references indexed in Scilit:
- Mind meets machine: Unravelling GPT-4’s cognitive psychologyBenchCouncil Transactions on Benchmarks, Standards and Evaluations, 2023
- Grounding the Vector Space of an Octopus: Word Meaning from Raw TextMinds and Machines, 2023
- A systematic evaluation of large language models of code2022
- A counterfactual simulation model of causal judgments for physical events.Psychological Review, 2021
- Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm2021
- PopMAG2020
- Building machines that learn and think like peopleBehavioral and Brain Sciences, 2016
- Causal Responsibility and CounterfactualsCognitive Science, 2013
- Causality2009
- The card game setThe Mathematical Intelligencer, 2003