Learning Scene Context Without Images
Preprint
Abstract
Teaching machines of scene contextual knowledge would enable them to interact more effectively with the environment and to anticipate or predict objects that may not be immediately apparent in their perceptual field. In this paper, we introduce a novel transformer-based approach called $LMOD$ ( Label-based Missing Object Detection) to teach scene contextual knowledge to machines using an attention mechanism. A distinctive aspect of the proposed approach is its reliance solely on labels from image datasets to teach scene context, entirely eliminating the need for the actual image itself. We show how scene-wide relationships among different objects can be learned using a self-attention mechanism. We further show that the contextual knowledge gained from label based learning can enhance performance of other visual based object detection algorithm.
Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- Context understanding in computer vision: A surveyComputer Vision and Image Understanding, 2023
- Occlusion-aware spatial attention transformer for occluded object recognitionPattern Recognition Letters, 2022
- Knowledge-Guided Multi-Label Few-Shot Learning for General Image RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Getting to know low-light images with the Exclusively Dark datasetComputer Vision and Image Understanding, 2019
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image AnnotationsInternational Journal of Computer Vision, 2017
- Neural Machine Translation of Rare Words with Subword Units2016