VOGUE: A Multimodal Dataset for Conversational Recommendation in Fashion
Preprint
Abstract
Multimodal conversational recommendation has emerged as a promising paradigm for delivering personalized experiences through natural dialogue enriched by visual and contextual grounding. Yet, current multimodal conversational recommendation datasets remain limited: existing resources either simulate conversations, omit user history, or fail to collect sufficiently detailed feedback, all of which constrain the types of research and evaluation they support. To address these gaps, we introduce VOGUE, a novel dataset of 60 humanhuman dialogues in realistic fashion shopping scenarios. Each dialogue is paired with a shared visual catalogue, item metadata, user fashion profiles and histories, and post-conversation ratings from both Seekers and Assistants. This design enables rigorous evaluation of conversational inference, including not only alignment between predicted and ground-truth preferences, but also calibration against full rating distributions and comparison with explicit and implicit user satisfaction signals. Our initial analyses of VOGUE reveal distinctive dynamics of visually grounded dialogue. For example, recommenders frequently suggest items simultaneously in feature-based groups, which creates distinct conversational phases bridged by Seeker critiques and refinements. Benchmarking multimodal large language models against human recommenders shows that while MLLMs approach human-level alignment in aggregate, they exhibit systematic distribution errors in reproducing human ratings and struggle to generalize preference inference beyond explicitly discussed items. These findings establish VOGUE as both a unique resource for studying multimodal conversational systems and as a challenge dataset beyond the current recommendation capabilities of existing top-tier multimodal foundation models such as GPT-4o-mini, GPT-5-mini, and Gemini-2.5-Flash.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- CoSRec: A Joint Conversational Search and Recommendation Dataset2025
- Large Language Models for Recommendation: Progresses and Future Directions2024
- MG-ShopDial: A Multi-Goal Conversational Dataset for e-Commerce2023
- MMConv: An Environment for Multimodal Conversational Search across Multiple Domains2021
- A Survey on Conversational Recommender SystemsACM Computing Surveys, 2021
- A Workflow Analysis of Context-driven Conversational Recommendation2021
- Advances and challenges in conversational recommender systems: A surveyAI Open, 2021
- Predicting User Intents and Satisfaction with Dialogue-based Conversational Recommendations2020
- Towards Conversational Recommender Systems2016
- Being accurate is not enough2006