AdaGlimpse: Active Visual Exploration with Arbitrary Glimpse Position and Scale
Preprint
Abstract
Active Visual Exploration (AVE) is a task that involves dynamically selecting observations (glimpses), which is critical to facilitate comprehension and navigation within an environment. While modern AVE methods have demonstrated impressive performance, they are constrained to fixed-scale glimpses from rigid grids. In contrast, existing mobile platforms equipped with optical zoom capabilities can capture glimpses of arbitrary positions and scales. To address this gap between software and hardware capabilities, we introduce AdaGlimpse. It uses Soft Actor-Critic, a reinforcement learning algorithm tailored for exploration tasks, to select glimpses of arbitrary position and scale. This approach enables our model to rapidly establish a general awareness of the environment before zooming in for detailed analysis. Experimental results demonstrate that AdaGlimpse surpasses previous methods across various visual tasks while maintaining greater applicability in realistic AVE scenarios.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Beyond Grids: Exploring Elastic Input Sampling for Vision Transformers2025
- FlexiViT: One Model for All Patch Sizes2023
- Consistency driven Sequential Transformers Attention Model for Partially Observable Scenes2022
- Omnivore: A Single Model for Many Visual Modalities2022
- Masked Autoencoders Are Scalable Vision Learners2022
- Glimpse-Attend-and-Explore: Self-Attention for Active Visual Exploration2021
- Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers2021
- The Change in the Altitude of an Unmanned Aerial Vehicle, Depending on the Height Difference of the Area TakenIOP Conference Series: Earth and Environmental Science, 2019
- Microsoft COCO: Common Objects in Context2014
- Eye movements in natural behaviorTrends in Cognitive Sciences, 2005