VisKE: Visual knowledge extraction and question answering by visual verification of relation phrases
Authors:
- 1 June 2015
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636919,p. 1456-1464
Abstract
How can we know whether a statement about our world is valid. For example, given a relationship between a pair of entities e.g., `eat(horse, hay)', how can we know whether this relationship is true or false in general. Gathering such knowledge about entities and their relationships is one of the fundamental challenges in knowledge extraction. Most previous works on knowledge extraction have focused purely on text-driven reasoning for verifying relation phrases. In this work, we introduce the problem of visual verification of relation phrases and developed a Visual Knowledge Extraction system called VisKE. Given a verb-based relation phrase between common nouns, our approach assess its validity by jointly analyzing over text and images and reasoning about the spatial consistency of the relative configurations of the entities and the relation involved. Our approach involves no explicit human supervision thereby enabling large-scale analysis. Using our approach, we have already verified over 12000 relation phrases. Our approach has been used to not only enrich existing textual knowledge bases by improving their recall, but also augment open-domain question-answer reasoning.Keywords
This publication has 27 references indexed in Scilit:
- Learning Knowledge Graphs for Question Answering through Conversational Dialog2015
- Adopting Abstract Images for Semantic Scene UnderstandingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Open question answering over curated and extracted knowledge bases2014
- Learning to Solve Arithmetic Word Problems with Verb Categorization2014
- Detecting Avocados to Zucchinis: What Have We Done, and Where Are We Going?2013
- NEIL: Extracting Visual Knowledge from Web Data2013
- A study of the knowledge base requirements for passing an elementary science test2013
- Acquiring temporal constraints between relations2012
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- Labeling images with a computer game2004