Investigating Navigation Strategies in the Morris Water Maze through Deep Reinforcement Learning
Preprint
Abstract
Navigation is a complex skill with a long history of research in animals and humans. In this work, we simulate the Morris Water Maze in 2D to train deep reinforcement learning agents. We perform automatic classification of navigation strategies, analyze the distribution of strategies used by artificial agents, and compare them with experimental data to show similar learning dynamics as those seen in humans and rodents. We develop environment-specific auxiliary tasks and examine factors affecting their usefulness. We suggest that the most beneficial tasks are potentially more biologically feasible for real agents to use. Lastly, we explore the development of internal representations in the activations of artificial agent neural networks. These representations resemble place cells and head-direction cells found in mouse brains, and their presence has correlation to the navigation strategies that artificial agents employ.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Effects of home environment structure on navigation preference and performance: A comparison in Veneto, Italy and Utah, USAJournal of Environmental Psychology, 2021
- Target-driven visual navigation in indoor scenes using deep reinforcement learning2017
- Variants of the Morris water maze task to comparatively assess human and rodent place navigationNeurobiology of Learning and Memory, 2017
- Place Cells, Grid Cells, and MemoryCold Spring Harbor Perspectives in Biology, 2015
- On the Properties of Neural Machine Translation: Encoder–Decoder Approaches2014
- Search strategies in a human water maze analogue analyzed with automatic classification methodsBehavioural Brain Research, 2010
- Spatial deficits in a virtual water maze in amnesic participants with hippocampal damageHippocampus, 2009
- Adult-Generated Hippocampal Neurons Allow the Flexible Use of Spatially Precise Learning StrategiesPLOS ONE, 2009
- Maze procedures: the radial-arm and water maze comparedCognitive Brain Research, 1996
- Developments of a water-maze procedure for studying spatial learning in the ratJournal of Neuroscience Methods, 1984