Reinforcement learning of a multi-link swimmer at low Reynolds numbers
- 1 March 2023
- journal article
- conference paper
- Published by AIP Publishing in Physics of Fluids
- Vol. 35 (3) , 032003
Abstract
The use of machine learning techniques in the development of microscopic swimmers has drawn considerable attention in recent years. In particular, reinforcement learning has been shown useful in enabling swimmers to learn effective propulsion strategies through its interactions with the surroundings. In this work, we apply a reinforcement learning approach to identify swimming gaits of a multi-link model swimmer. The swimmer consists of multiple rigid links connected serially with hinges, which can rotate freely to change the relative angles between neighboring links. Purcell [“Life at low Reynolds number,” Am. J. Phys. 45, 3 (1977)] demonstrated how the particular case of a three-link swimmer (now known as Purcell's swimmer) can perform a prescribed sequence of hinge rotation to generate self-propulsion in the absence of inertia. Here, without relying on any prior knowledge of low-Reynolds-number locomotion, we first demonstrate the use of reinforcement learning in identifying the classical swimming gaits of Purcell's swimmer for case of three links. We next examine the new swimming gaits acquired by the learning process as the number of links increases. We also consider the scenarios when only a single hinge is allowed to rotate at a time and when simultaneous rotation of multiple hinges is allowed. We contrast the difference in the locomotory gaits learned by the swimmers in these scenarios and discuss their propulsion performance. Taken together, our results demonstrate how a simple reinforcement learning technique can be applied to identify both classical and new swimming gaits at low Reynolds numbers.Keywords
Funding Information
- National Science Foundation (1830958)
- National Science Foundation (1931292)
- Singapore Ministry of Education Academic Research Fund Tier 2 grant (MOE-T2EP50221-0012)
- National University of Singapore Startup Grant (R-265-000-696-133)
- A*Star Advanced Manufacturing and Engineering Young Individual Research Grants (A2084c0175)
This publication has 42 references indexed in Scilit:
- Reverse engineering the euglenoid movementProceedings of the National Academy of Sciences, 2012
- Microrobots for Minimally Invasive MedicineAnnual Review of Biomedical Engineering, 2010
- The hydrodynamics of swimming microorganismsReports on Progress in Physics, 2009
- Optimal Stroke Patterns for Purcell’s Three-Link SwimmerPhysical Review Letters, 2007
- BIOFLUIDMECHANICS OF REPRODUCTIONAnnual Review of Fluid Mechanics, 2006
- Purcell’s “rotator”: mechanical rotation at low Reynolds numberThe European Physical Journal B, 2005
- Simple swimmer at low Reynolds number: Three linked spheresPhysical Review E, 2004
- On self-propulsion of micro-machines at low Reynolds number: Purcell's three-link swimmerJournal of Fluid Mechanics, 2003
- Fluid Mechanics of Propulsion by Cilia and FlagellaAnnual Review of Fluid Mechanics, 1977
- Analysis of the swimming of microscopic organismsProceedings of the Royal Society of London. A. Mathematical and Physical Sciences, 1951