Sacrifice One For the Good of Many?
Open Access
- 2 March 2015
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 117-124
Abstract
Moral norms play an essential role in regulating human interaction. With the growing sophistication and proliferation of robots, it is important to understand how ordinary people apply moral norms to robot agents and make moral judgments about their behavior. We report the first comparison of people's moral judgments (of permissibility, wrongness, and blame) about human and robot agents. Two online experiments (total N = 316) found that robots, compared with human agents, were more strongly expected to take an action that sacrifices one person for the good of many (a "utilitarian" choice), and they were blamed more than their human counterparts when they did not make that choice. Though the utilitarian sacrifice was generally seen as permissible for human agents, they were blamed more for choosing this option than for doing nothing. These results provide a first step toward a new field of Moral HRI, which is well placed to help guide the design of social robots.Keywords
Funding Information
- Office of Naval Research (N00014-14-l-0144)
This publication has 33 references indexed in Scilit:
- Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral ResearchPLOS ONE, 2013
- A Conceptual and Computational Model of Moral Decision Making in Human and Artificial AgentsTopics in Cognitive Science, 2010
- Crime and punishment: Distinguishing the roles of causal and intentional analyses in moral judgmentCognition, 2008
- Deciding versus Reacting: Conceptions of Moral Judgment and the Reason-Affect DebateReview of General Psychology, 2007
- The Role of Conscious Reasoning and Intuition in Moral JudgmentPsychological Science, 2006
- Blaming Agents in Moral DilemmasEthical Theory and Moral Practice, 2006
- Toward a General Logicist Methodology for Engineering Ethically Correct RobotsIEEE Intelligent Systems, 2006
- Editorial: Ethics and Engineering DesignScience, Technology, & Human Values, 2006
- An fMRI Investigation of Emotional Engagement in Moral JudgmentScience, 2001
- The Trolley ProblemThe Yale Law Journal, 1985