The Voice: Lessons on Trustworthy Conversational Agents from 'Dune'
Open Access
- 8 July 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-5
Abstract
The potential for untrustworthy conversational agents presents a significant threat for covert social manipulation. Taking inspiration from Frank Herbert’s Dune [12], where the Bene Gesserit Sisterhood uses the Voice for influence, manipulation, and control of people, we explore how generative AI provides a way to implement individualized influence at industrial scales. Already, these models can manipulate communication across text, image, speech, and most recently video. They are rapidly becoming affordable enough for any organization of even moderate means to train and deploy. If employed by malicious actors, they risk becoming powerful tools for shaping public opinion, sowing discord, and undermining organizations from companies to governments. As researchers and developers, it is crucial to recognize the potential for such weaponization and to explore strategies for prevention, detection, and defense against these emerging forms of sociotechnical manipulation.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Distorting the truth versus blatant lies: The effects of different degrees of deception in domestic and foreign political deepfakesComputers in Human Behavior, 2023
- Metaphors in Voice User Interfaces: A Slippery FishACM Transactions on Computer-Human Interaction, 2023
- Transitional justice and online social platforms: Facebook and the Rohingya genocideInternational Journal of Law and Information Technology, 2023
- Exploring Deceptive Design Patterns in Voice Interfaces2022
- ”I am Definitely Manipulated, Even When I am Aware of it. It’s Ridiculous!” - Dark Patterns from the End-User Perspective2021
- Tea, Earl Grey, Hot: Designing Speech Interactions from the Imagined Ideal of Star Trek2021
- Voice pitch modulation in human mate choiceProceedings Of The Royal Society B-Biological Sciences, 2018
- Make it so2012
- Sounds like a winner: voice pitch influences perception of leadership capacity in both men and womenProceedings Of The Royal Society B-Biological Sciences, 2012
- The Sociolinguistic Significance of Conversational Code-SwitchingRELC Journal, 1977