On Available Corpora for Empirical Methods in Vision & Language

Authors:
  • FFFrancis Ferraro
    Francis Ferraro
  • NMNasrin Mostafazadeh
    Nasrin Mostafazadeh
  • THTing-Hao Huang
    Ting-Hao Huang
  • LVLucy Vanderwende
    Lucy Vanderwende
  • JDJacob Devlin
    Jacob Devlin
  • MGMichel Galley
    Michel Galley
  • MMMargaret Mitchell
    Margaret Mitchell

Abstract
Integrating vision and language has long been a dream in work on artificial intelligence (AI). In the past two years, we have witnessed an explosion of work that brings together vision and language from images to videos and beyond. The available corpora have played a crucial role in advancing this area of research. In this paper, we propose a set of quality metrics for evaluating and analyzing the vision & language datasets and classify them accordingly. Our analyses show that the most recent datasets have been using more complex language and more abstract concepts, however, there are different strengths and weaknesses in each.