PicPersona-TOD : A Dataset for Personalizing Utterance Style in Task-Oriented Dialogue with Image Persona
Preprint
Abstract
Task-Oriented Dialogue (TOD) systems are designed to fulfill user requests through natural language interactions, yet existing systems often produce generic, monotonic responses that lack individuality and fail to adapt to users' personal attributes. To address this, we introduce PicPersona-TOD, a novel dataset that incorporates user images as part of the persona, enabling personalized responses tailored to user-specific factors such as age or emotional context. This is facilitated by first impressions, dialogue policy-guided prompting, and the use of external knowledge to reduce hallucinations. Human evaluations confirm that our dataset enhances user experience, with personalized responses contributing to a more engaging interaction. Additionally, we introduce a new NLG model, Pictor, which not only personalizes responses, but also demonstrates robust performance across unseen domains https://github.com/JihyunLee1/PicPersona.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- MEDIC: A Multimodal Empathy Dataset in Counseling2023
- Dialogue State Tracking with a Language Model using Schema-Driven Prompting2021
- Adding Chit-Chat to Enhance Task-Oriented Dialogues2021
- Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems2021
- Alexa, do voice assistants influence consumer brand engagement? – Examining the role of AI powered voice assistants in influencing consumer brand engagementJournal of Business Research, 2021
- Towards Persona-Based Empathetic Conversational Models2020
- GoEmotions: A Dataset of Fine-Grained Emotions2020
- Personal Greetings: Personalizing Robot Utterances Based on Novelty of Observed BehaviorInternational Journal of Social Robotics, 2016
- POMDP-Based Statistical Spoken Dialog Systems: A ReviewProceedings of the IEEE, 2013
- Extraversion is accurately perceived after a 50-ms exposure to a faceJournal of Research in Personality, 2009