Samuel Rhys Cox

Large language model (LLM)-based chatbots offer new methodological opportunities for psychological research by enabling scalable, standardised, and interactive research procedures. However, their usefulness depends on the researchers’ ability to customise, evaluate, and deploy them in ways that align with the methodological requirements of a given study. The current tutorial provides a practical, step-by-step framework for developing customised LLM-based chatbots for psychological research. We distinguish between no-code or low-code development approaches, including provider-hosted chatbot builders and visual or low-code programming platforms, and code-based approaches using proprietary model application programming interfaces (APIs) or locally hosted open-weight models. We compare these approaches in terms of technical expertise, behavioural and workflow control, study integration, cost, data governance, maintenance, and reproducibility. We also outline key components of the process of customising an LLM-based chatbot and illustrate deployment workflows across multiple platforms, including local desktop interfaces, messaging applications and survey platforms. Finally, we discuss key considerations for research ethics, privacy, data governance, security, transparency, and reproducibility, including recommendations for documenting and sharing chatbot configurations and study procedures. Through this tutorial, we aim to support researchers in integrating customised LLM-based chatbots into psychological research and in expanding the methodological possibilities available to them.

As conversational agents become increasingly common in behaviour change interventions, understanding optimal feedback delivery mechanisms becomes increasingly important. However, choosing a style that both lessens psychological reactance (perceived threats to freedom) while simultaneously eliciting feelings of surprise and engagement represents a complex design problem. We explored how three different feedback styles: Direct, Politeness, and Verbal Leakage (slips or disfluencies to reveal a desired behaviour) affect user perceptions and behavioural intentions. Matching expectations from literature, the Direct chatbot led to lower behavioural intentions and higher reactance, while the Politeness chatbot evoked higher behavioural intentions and lower reactance. However, Politeness was also seen as unsurprising and unengaging by participants. In contrast, Verbal Leakage evoked reactance, yet also elicited higher feelings of surprise, engagement, and humour. These findings highlight that effective feedback requires navigating trade-offs between user reactance and engagement, with novel approaches such as Verbal Leakage offering promising alternative design opportunities.

People increasingly turn to conversational agents such as ChatGPT to seek guidance for their personal problems. As these systems grow in capability, many now display elements of "thinking": short reflective statements that reveal a model's intentions or values before responding. While initially introduced to promote transparency, such visible thinking can also anthropomorphise the agent and shape user expectations. Yet little is known about how these displays affect user perceptions in help-seeking contexts. We conducted a 3 x 2 mixed design experiment examining the impact of 'Thinking Content' (None, Emotionally-Supportive, Expertise-Supportive) and 'Conversation Context' (Habit-related vs. Feelings-related problems) on users' perceptions of empathy, warmth, competence, and engagement. Participants interacted with a chatbot that either showed no visible thinking or presented value-oriented reflections prior to its response. Our findings contribute to understanding how thinking transparency influences user experience in supportive dialogues, and offer implications for designing conversational agents that communicate intentions in sensitive, help-seeking scenarios.

Self-disclosure is important to help us feel better, yet is often difficult. This difficulty can arise from how we think people are going to react to our self-disclosure. In this workshop paper, we briefly discuss self-disclosure to conversational user interfaces (CUIs) in relation to various social cues. We then, discuss how expressions of uncertainty or representation of a CUI's reasoning could help encourage self-disclosure, by making a CUI's intended "theory of mind" more transparent to users.

In this workshop paper, we discuss the potential for measures of user-centric benefits (such as emotional well-being) that could be explored when evaluating explainable AI (XAI) systems within the arts. As a background to this, we draw from our recent review of creativity support tool (CST) evaluations, that found a paucity of studies evaluating CSTs for user-centric measures that benefit the user themselves. Specifically, we discuss measures of: (1) developing intrinsic abilities, (2) emotional well-being, (3) self-reflection, and (4) self-perception. By discussing these user-centric measures within the context of XAI and the arts, we wish to provoke discussion regarding the potential of such measures.