Understanding Users' Privacy Reasoning and Behaviors During Chatbot Use to Support Meaningful Agency in Privacy
Authors:
- Mohammad Hadi NezhadMNMohammad Hadi Nezhad
- Francisco Enrique Vicente CastroFCFrancisco Enrique Vicente Castro
- Ivon ArroyoIAIvon Arroyo
Preprint
Abstract
Conversational agents (CAs) (e.g., chatbots) are increasingly used in settings where users disclose sensitive information, raising significant privacy concerns. Because privacy judgments are highly contextual, supporting users to engage in privacy-protective actions during chatbot interactions is essential. However, enabling meaningful engagement requires a deeper understanding of how users currently reason about and manage sensitive information during realistic chatbot use scenarios. To investigate this, we qualitatively examined computer science (undergraduate and masters) students' in-the-moment disclosure and protection behaviors, as well as the reasoning underlying these behaviors, across a range of realistic chatbot tasks. Participants used a simulated ChatGPT interface with and without a privacy notice panel that intercepts message submissions, highlights potentially sensitive information, and offers privacy protective actions. The panel supports anonymization through retracting, faking, and generalizing, and surfaces two of ChatGPT's built-in privacy controls to improve their discoverability. Drawing on interaction logs, think-alouds, and survey responses, we analyzed how the panel fostered privacy awareness, encouraged protective actions, and supported context-specific reasoning about what information to protect and how. We further discuss design opportunities for tools that provide users greater and more meaningful agency in protecting sensitive information during CA interactions.Keywords
All Related Versions
This publication has 36 references indexed in Scilit:
- Embedding Ethical Awareness in Computer Science and AI Education: The PEaRCE Approach to Responsible Computing2025
- Exploring artificial intelligence (AI) Chatbot usage behaviors and their association with mental health outcomes in Chinese university studentsJournal of Affective Disorders, 2025
- “Always check important information!” - The role of disclaimers in the perception of AI-generated contentComputers in Human Behavior: Artificial Humans, 2025
- Math Teachers’ In-Class Information Needs and Usage for Effective Design of Classroom Orchestration Tools2024
- Perceptions and usage of AI chatbots among students in higher education across genders, academic levels and fields of studyComputers and Education: Artificial Intelligence, 2024
- Temporal dynamics of decision making: A synthesis of computational and neurophysiological approachesWIREs Cognitive Science, 2021
- Privacy Concerns in Chatbot Interactions2020
- Privacy by DesignIdentity in the Information Society, 2010
- Self‐disclosure in computer‐mediated communication: The role of self‐awareness and visual anonymityEuropean Journal of Social Psychology, 2001
- Information displays and preference reversalsOrganizational Behavior and Human Decision Processes, 1988