Whose Wisdom? Human Biases for Decision Support System Source and Scale
- 9 August 2025
- journal article
- Published by SAGE Publications in Proceedings of the Human Factors and Ergonomics Society Annual Meeting
- Vol. 69 (1) , 2032-2037
Abstract
Users form snap judgments about decision‑support systems (DSS) that guide later reliance. We examined how those judgments depend on DSS source—machine‑learning algorithm (ML) versus wisdom‑of‑the‑crowd (WoC)—and data scale (small, medium, large). Seventy‑eight participants repeatedly chose between paired advisers (e.g., medium‑ML vs. large‑WoC) and then performed a six‑trial visual‑search task with help from their selected advisor. Participants showed no inherent bias for ML or WoC but consistently preferred advisers built on larger datasets. Within six trials, reliability estimates were calibrated to observed accuracy, indicating rapid trust adjustment. Findings suggest users intuitively weighed sample size over provenance when forecasting DSS performance and that well‑sized crowd systems can compete with contemporary ML advisers.Keywords
This publication has 13 references indexed in Scilit:
- Elicitation and aggregation of multimodal estimates improve wisdom of crowd effects on ordering tasksScientific Reports, 2024
- The (Im)perfect Automation Schema: Who Is Trusted More, Automated or Human Decision Support?Human Factors: The Journal of the Human Factors and Ergonomics Society, 2023
- Preference for human, not algorithm aversionTrends in Cognitive Sciences, 2022
- Who's the real expert here? Pedigree's unique bias on trust between human and automated advisersApplied Ergonomics, 2019
- Algorithm appreciation: People prefer algorithmic to human judgmentOrganizational Behavior and Human Decision Processes, 2019
- Revisiting Unreasonable Effectiveness of Data in Deep Learning Era2017
- Impact of Training and Testing Data Splits on Accuracy of Time Series Forecasting in Machine Learning2017
- Algorithm aversion: People erroneously avoid algorithms after seeing them err.Journal of Experimental Psychology: General, 2015
- Individual Differences in the Calibration of Trust in AutomationHuman Factors: The Journal of the Human Factors and Ergonomics Society, 2014
- Automation bias: a systematic review of frequency, effect mediators, and mitigatorsJournal of the American Medical Informatics Association, 2012