Mapping the Wizards' Path: A Systematic Review of Wizard-of-Oz in HCI
Open Access
- 13 April 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-20
Abstract
The Wizard-of-Oz (WoZ) method has long been a core prototyping technique in Human-Computer Interaction (HCI), in which users interact with systems that seem autonomous but are actually controlled by hidden human operators. Advances in interactive technologies have expanded the landscape of future system behaviors, broadening both where and how WoZ is used. However, as more envisioned behaviors become technically feasible, the distinction between engineering a system and simulating an interaction becomes blurred, making it essential to clarify when and why to employ wizarding. This paper presents the first systematic review of WoZ in HCI, drawing on 194 papers from SIGCHI venues to identify ten application domains, five wizard control types, eight motivations, and five categories of concerns. Building on these findings, we propose a reciprocal evolution framework that interprets how technology and wizarding shape each other, and derive guidelines for the rigorous application of WoZ. We further illustrate the framework through emerging prototyping practices with Large Language Models (LLMs).Keywords
This publication has 172 references indexed in Scilit:
- Embodied large language models enable robots to complete complex tasks in unpredictable environmentsNature Machine Intelligence, 2025
- The rise and potential of large language model based agents: a surveyScience China Information Sciences, 2025
- K-Alpha Calculator–Krippendorff's Alpha Calculator: A user-friendly tool for computing Krippendorff's Alpha inter-rater reliability coefficientMethodsX, 2024
- Digital Human-in-the-Loop Framework2020
- AI-Mediated Communication: Definition, Research Agenda, and Ethical ConsiderationsJournal of Computer-Mediated Communication, 2020
- Qualitative research: deductive and inductive approaches to data analysisQualitative Research Journal, 2018
- Wizard of Oz Experimentation for Language Technology Applications: Challenges and ToolsInteracting with Computers, 2014
- Adapting Performance Feedback to a Learner’s ConscientiousnessLecture Notes in Computer Science, 2012
- What do Prototypes Prototype?1997
- Simulating speech systemsComputer Speech & Language, 1991