Conversational AI and Transparency Statements: Design Options and Impact on User Perceptions

Authors:
  • AFAsbjørn Følstad
    Asbjørn Følstad
  • KKKnut Kvale
    Knut Kvale
  • ALAnna Grøndal Larsen
    Anna Grøndal Larsen
  • NANena van As
    Nena van As

Abstract
Conversational AI powered by large language models is increasingly seen as an attractive technology for customer service due to its ability to provide fluent, efficient support. However, the technology's humanlike qualities can make it hard for users to realize they're interacting with an AI rather than a human employee, and the unpredictability of LLMs increases the risk of misleading or inaccurate outputs. This creates a conversational AI transparency design challenge: How can we disclose conversational AI use and communicate its limitations in ways that users notice and understand? To address this, we conducted two complementary studies examining user perceptions and the effects of transparency statements (disclosures and disclaimers) in customer service contexts. In Study 1 (N = 300), participants evaluated multiple transparency design options presented as chat screenshots, varying statement wording and placement (e.g., top/bottom of the interface vs. message-level labels with and without tooltips). The results indicate that participants' preferences were driven primarily by visual prominence and low interaction cost; clear wording at the top or bottom of the interface was often seen as beneficial, tooltipbased designs were seen as easy to miss, and selective message-level labeling risked confusing users about which messages were AI-generated. In Study 2 (N = 208), participants interacted with a GPT4o-powered conversational AI prototype under three conditions (no disclosure, disclosure only, disclosure and disclaimer). The results show that many participants did not notice or retain the transparency statements, and we found no significant differences across conditions in regard to trust, willingness to use, or likelihood of following advice. Together, the results suggest that while transparency statements are necessary to comply with transparency requirements and have been thought to be beneficial for trust calibration, designing transparency statements that users notice and remember remains an important design challenge.