Human Perception of AI Capabilities at Classifying Perturbed Roadway Signs
Open Access
- 26 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Human-Machine Systems
- Vol. 55 (4) , 499-507
Abstract
Artificial Intelligence (AI) is crucial to numerous functions required for driving automation systems, including the computer vision techniques used to detect the roadway environment and make real-time decisions. However, the images used as inputs to the AI system may be maliciously perturbed, or manipulated, causing the AI system to make an incorrect classification. In this study, we examined humans’ perception of the AI’s computer vision capability of classifying various road sign images, including the original images, images with two different types of malicious attacks, and images that are scrambled randomly at the pixel level. Our results showed that participants rated the AI agent to be less capable than themselves of classifying the road signs. However, they overestimated the AI’s computer vision capability for correctly classifying images with malicious attacks that should cause the AI system to misclassify the image. These findings suggest that people lack an accurate understanding of the vulnerabilities of AI computer vision technologies and tend to overtrust AI in driving automation systems.Keywords
Funding Information
- National Science Foundation (#2007386, #2245055)
This publication has 39 references indexed in Scilit:
- Traffic Sign Recognition Robustness in Autonomous Vehicles Under Physical Adversarial Attacks2023
- Artificial Intelligence, Logic, and Formalising Common Sense2022
- “Rogue machines” and crisis communication: When AI fails, how do companies publicly respond?Public Relations Review, 2021
- A critical review on computer vision and artificial intelligence in food industryJournal of Agriculture and Food Research, 2020
- Autonomous Vehicles and Embedded Artificial Intelligence: The Challenges of Framing Machine Driving DecisionsApplied Artificial Intelligence, 2019
- Siri, Siri, in my hand: Who’s the fairest in the land? On the interpretations, illustrations, and implications of artificial intelligenceBusiness Horizons, 2019
- Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision makingBusiness Horizons, 2018
- Human Information Interaction, Artificial Intelligence, and Errors2017
- Artificial intelligence techniques for driving safety and vehicle crash predictionArtificial Intelligence Review, 2016
- A study on key technologies of unmanned drivingCAAI Transactions on Intelligence Technology, 2016