A Survey of Current Datasets for Vision and Language Research
Authors:
- Ting- HaoTHTing- Hao
- Lucy VanderwendeLVLucy Vanderwende
- Jacob DevlinJDJacob Devlin
- Michel GalleyMGMichel Galley
- Margaret MitchellMMMargaret Mitchell
Preprint
Abstract
Integrating vision and language has long been a dream in work on artificial intelligence (AI). In the past two years, we have witnessed an explosion of work that brings together vision and language from images to videos and beyond. The available corpora have played a crucial role in advancing this area of research. In this paper, we propose a set of quality metrics for evaluating and analyzing the vision & language datasets and categorize them accordingly. Our analyses show that the most recent datasets have been using more complex language and more abstract concepts, however, there are different strengths and weaknesses in each.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- VQA: Visual Question AnsweringInternational Journal of Computer Vision, 2016
- Long-Term Recurrent Convolutional Networks for Visual Recognition and DescriptionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Visual Madlibs: Fill in the Blank Description Generation and Question Answering2015
- From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptionsTransactions of the Association for Computational Linguistics, 2014
- Microsoft COCO: Common Objects in Context2014
- Training a Multilingual Sportscaster: Using Perceptual Context to Learn LanguageJournal of Artificial Intelligence Research, 2010