A Survey on Transformers in Reinforcement Learning
Preprint
Abstract
Transformer has been considered the dominating neural architecture in NLP and CV, mostly under supervised settings. Recently, a similar surge of using Transformers has appeared in the domain of reinforcement learning (RL), but it is faced with unique design choices and challenges brought by the nature of RL. However, the evolution of Transformers in RL has not yet been well unraveled. In this paper, we seek to systematically review motivations and progress on using Transformers in RL, provide a taxonomy on existing works, discuss each sub-field, and summarize future prospects.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Multi-agent Multi-game Entity Transformer: Towards Generalist Models in MARL2024
- RT-1: Robotics Transformer for Real-World Control at Scale2023
- Meta-Reinforcement Learning via Language Instructions2023
- Frozen Pretrained Transformers as Universal Computation EnginesProceedings of the AAAI Conference on Artificial Intelligence, 2022
- CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learning2022
- Context-Aware Language Modeling for Goal-Oriented Dialogue Systems2022
- Mastering Atari, Go, chess and shogi by planning with a learned modelNature, 2020
- Grandmaster level in StarCraft II using multi-agent reinforcement learningNature, 2019
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998