Neuromorphic sequence learning with an event camera on routes through vegetation
- 13 September 2023
- journal article
- research article
- Published by American Association for the Advancement of Science (AAAS) in Science Robotics
- Vol. 8 (82) , eadg3679
Abstract
For many robotics applications, it is desirable to have relatively low-power and efficient onboard solutions. We took inspiration from insects, such as ants, that are capable of learning and following routes in complex natural environments using relatively constrained sensory and neural systems. Such capabilities are particularly relevant to applications such as agricultural robotics, where visual navigation through dense vegetation remains a challenging task. In this scenario, a route is likely to have high self-similarity and be subject to changing lighting conditions and motion over uneven terrain, and the effects of wind on leaves increase the variability of the input. We used a bioinspired event camera on a terrestrial robot to collect visual sequences along routes in natural outdoor environments and applied a neural algorithm for spatiotemporal memory that is closely based on a known neural circuit in the insect brain. We show that this method is plausible to support route recognition for visual navigation and more robust than SeqSLAM when evaluated on repeated runs on the same route or routes with small lateral offsets. By encoding memory in a spiking neural network running on a neuromorphic computer, our model can evaluate visual familiarity in real time from event camera footage.Keywords
This publication has 102 references indexed in Scilit:
- Spatial Memory in Insect NavigationCurrent Biology, 2013
- Visual homing: an insect perspectiveCurrent Opinion in Neurobiology, 2012
- A Model of Ant Route Navigation Driven by Scene FamiliarityPLOS Computational Biology, 2012
- Holistic visual encoding of ant-like routes: Navigation without waypointsAdaptive Behavior, 2011
- Sparse odor representation and olfactory learningNature Neuroscience, 2008
- Phenomenological models of synaptic plasticity based on spike timingBiological Cybernetics, 2008
- Idiosyncratic route-based memories in desert ants, Melophorus bagoti: How do they interact with path-integration vectors?Neurobiology of Learning and Memory, 2005
- Oscillations and Sparsening of Odor Representations in the Mushroom BodyScience, 2002
- VISUAL ACUITY IN INSECTSAnnual Review of Entomology, 1997
- The dynamics of phototransduction in insectsJournal of Comparative Physiology A, 1984