Emergence of Chemotactic Strategies with Multi-Agent Reinforcement Learning
Preprint
Abstract
Reinforcement learning (RL) is a flexible and efficient method for programming micro-robots in complex environments. Here we investigate whether reinforcement learning can provide insights into biological systems when trained to perform chemotaxis. Namely, whether we can learn about how intelligent agents process given information in order to swim towards a target. We run simulations covering a range of agent shapes, sizes, and swim speeds to determine if the physical constraints on biological swimmers, namely Brownian motion, lead to regions where reinforcement learners' training fails. We find that the RL agents can perform chemotaxis as soon as it is physically possible and, in some cases, even before the active swimming overpowers the stochastic environment. We study the efficiency of the emergent policy and identify convergence in agent size and swim speeds. Finally, we study the strategy adopted by the reinforcement learning algorithm to explain how the agents perform their tasks. To this end, we identify three emerging dominant strategies and several rare approaches taken. These strategies, whilst producing almost identical trajectories in simulation, are distinct and give insight into the possible mechanisms behind which biological agents explore their environment and respond to changing conditions.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Multi-agent deep reinforcement learning: a surveyArtificial Intelligence Review, 2021
- A Concise Introduction to Decentralized POMDPs2016
- Quorum sensing enhancement of the stress response promotes resistance to quorum quenching and prevents social cheatingThe ISME Journal, 2014
- The Hydrodynamics of a Run-and-Tumble Bacterium Propelled by Polymorphic Helical FlagellaBiophysical Journal, 2010
- Making sense of it all: bacterial chemotaxisNature Reviews Molecular Cell Biology, 2004
- Stokes drag on axially symmetric bodies: a new approachProceedings - Mathematical Sciences, 1999
- Modification of the overlap potential to mimic a linear site–site potentialThe Journal of Chemical Physics, 1981
- Brownian motion of an ellipsoid. A correction to Perrin's resultsBiopolymers, 1975
- Chemotaxis in Escherichia coli analysed by Three-dimensional TrackingNature, 1972
- Role of Repulsive Forces in Determining the Equilibrium Structure of Simple LiquidsThe Journal of Chemical Physics, 1971