AI Persona Design and Perceived Difficulty of Legal Advice Tasks

Authors:
Preprint
Publisher website
Google Scholar
Abstract
Personality cues may influence how users experience conversational artificial intelligence, but there is currently limited understanding of their role in high-risk situations involving expert advice. We developed and tested an experimental approach in which the occupational character of a legal advice chatbot was varied and looked at how this affected users' perceptions of task difficulty in relation to their Big Five personality traits. Sixteen adults (eight women and eight men; mean age 26.50 years, standard deviation 4.32) were randomly given one of three personas: judge (n = 5), lawyer (n = 5), or mediator (n = 6). Each participant worked through three randomly selected legal scenarios and then rated the difficulty of the tasks. The Big Five traits were measured using the BFI-44; emotion regulation (FEEL-E) was also used as an additional check on neuroticism-related patterns. There were strong differences between the groups in terms of the mean level of subjective difficulty, F(2, 13) = 33.60, p < .001, η² = .84, ω² = .80. The mediator persona was associated with the lowest rating of difficulty (M = 1.67, SD = 0.42), followed by the lawyer persona (M = 3.60, SD = 0.43) and then the judge persona (M = 4.07, SD = 0.68). Within the small subgroup sizes — ranging from five to six — exploratory Spearman correlations indicated different patterns depending on personality trait and persona, such as negative associations for the mediator and a strong positive association between conscientiousness and difficulty for the judge. Since the subgroup sizes were only five or six, these correlations are only hypothesis-generating and not confirmatory. The approach proved feasible and led to specific design improvements for a previously registered, adequately powered study. The evidence shows that using mediator-like cues — particularly high agreeableness and low neuroticism in the chatbot's persona — could improve perceptions of the understandability of complex advice.
Funding Information
  • Österreichische Forschungsförderungsgesellschaft (FO999915238)