Unsupervised Salient Patch Selection for Data-Efficient Reinforcement Learning
Preprint
Abstract
To improve the sample efficiency of vision-based deep reinforcement learning (RL), we propose a novel method, called SPIRL, to automatically extract important patches from input images. Following Masked Auto-Encoders, SPIRL is based on Vision Transformer models pre-trained in a self-supervised fashion to reconstruct images from randomly-sampled patches. These pre-trained models can then be exploited to detect and select salient patches, defined as hard to reconstruct from neighboring patches. In RL, the SPIRL agent processes selected salient patches via an attention module. We empirically validate SPIRL on Atari games to test its data-efficiency against relevant state-of-the-art methods, including some traditional model-based methods and keypoint-based models. In addition, we analyze our model's interpretability capabilities.Keywords
All Related Versions
- Version 1, 2024-01-10, ArXiv
- Published version: , 556.
This publication has 9 references indexed in Scilit:
- Pretraining the Vision Transformer Using Self-Supervised Methods for Vision Based Deep Reinforcement Learning2023
- Improving Sample Efficiency in Model-Free Reinforcement Learning from ImagesProceedings of the AAAI Conference on Artificial Intelligence, 2021
- Invariant Transform Experience Replay: Data Augmentation for Deep Reinforcement LearningIEEE Robotics and Automation Letters, 2020
- An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents2019
- Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General AgentsJournal of Artificial Intelligence Research, 2018
- Deep Reinforcement Learning with Double Q-LearningProceedings of the AAAI Conference on Artificial Intelligence, 2016
- The Arcade Learning Environment: An Evaluation Platform for General AgentsJournal of Artificial Intelligence Research, 2013
- Equitable aggregations and multiple criteria analysisEuropean Journal of Operational Research, 2004
- Markov Decision Processes: Discrete Stochastic Dynamic ProgrammingTechnometrics, 1995