Noisy Networks for Exploration
Authors:
- Meire FortunatoMFMeire Fortunato
- Mohammad Gheshlaghi AzarMAMohammad Gheshlaghi Azar
- Bilal PiotBPBilal Piot
- Jacob MenickJMJacob Menick
- Ian OsbandIOIan Osband
- Alex GravesAGAlex Graves
- Vlad MnihVMVlad Mnih
- Remi MunosRMRemi Munos
- Demis HassabisDHDemis Hassabis
- Olivier PietquinOPOlivier Pietquin
- Charles BlundellCBCharles Blundell
- Shane LeggSLShane Legg
Preprint
Abstract
We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent's policy can be used to aid efficient exploration. The parameters of the noise are learned with gradient descent along with the remaining network weights. NoisyNet is straightforward to implement and adds little computational overhead. We find that replacing the conventional exploration heuristics for A3C, DQN and dueling agents (entropy reward and $\epsilon$-greedy respectively) with NoisyNet yields substantially higher scores for a wide range of Atari games, in some cases advancing the agent from sub to super-human performance.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Kalman Temporal DifferencesJournal of Artificial Intelligence Research, 2010
- Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010)IEEE Transactions on Autonomous Mental Development, 2010
- What is intrinsic motivation? A typology of computational approachesFrontiers in Neurorobotics, 2007
- Flat MinimaNeural Computation, 1997
- Markov Decision Processes: Discrete Stochastic Dynamic ProgrammingTechnometrics, 1995
- Training with Noise is Equivalent to Tikhonov RegularizationNeural Computation, 1995