Victor Alfred Stimpfling
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Victor Alfred Stimpfling has a total of 8 co-authors. Here are the top co-authors by number of shared publications:
- Pavan Ramdya
EPFL
3 shared publications - Sibo Wang
Brain Mind Institute, cole Polytechnique Fdrale de Lausanne (EPFL), Lausanne, Switzerland
3 shared publications - Femke Hurtak
Neuroengineering Laboratory, Brain Mind Institute & Interfaculty Institute of Bioengineering, EPFL, Lausanne, Switzerland
2 shared publications Louise Genoud
Neuroengineering Laboratory, Brain Mind Institute & Interfaculty Institute of Bioengineering, EPFL, Lausanne, Switzerland
2 shared publications- Pembe Gizem Özdil
Neuroengineering Laboratory, Brain Mind Institute & Interfaculty Institute of Bioengineering, EPFL, Lausanne, Switzerland
2 shared publications - Maite Azcorra
Neuroengineering Laboratory, Brain Mind Institute & Institute of Bioengineering, EPFL, Lausanne, Switzerland
1 shared publication
Latest Publications
- Open Access
Precise kinematic and muscle recording in freely behaving flies enabled by closed-loop tracking and annotation-free pose estimation
Understanding the neuromuscular basis for behavior requires measuring both kinematic and physiological data at high resolution in unconstrained conditions: a technically challenging goal. Here we present an integrated experimental-computational pipeline for measuring and quantifying body part kinematics and muscle activity in freely behaving Drosophila melanogaster . We first present Spotlight , a closed-loop videography system that performs real-time tracking to record untethered flies at high resolution (6 µm/pixel) and high frame rate (330 Hz) while also enabling optical recordings of limb muscle activity via a fluorescent calcium reporter. To analyze these massive datasets without manual image annotation, we introduce PoseForge , a synthetic-data-driven framework that exploits morphologically accurate biomechanical simulations to generate synthetic data, and contrastive self-supervised learning to infer 3D keypoints and dense body-part segmentation from a single camera view. Using resulting 3D kinematic data, we can replay recorded behaviors in a biomechanical digital twin, NeuroMechFly, to infer forces generated and experienced by the fly’s limbs. Finally, we illustrate the capability of our system to optically record muscle activity. We show how the legs’ long-tendon muscles activate upon mechanical vibration, possibly to activate gripping and to maintain a stable posture. Taken together, this workflow enables scalable, high-resolution measurement and modeling of unconstrained, natural behavior.
Open AccessPrecise kinematic and muscle recording in freely behaving flies enabled by closed-loop tracking and annotation-free pose estimation
Understanding the neuromuscular basis for behavior requires measuring both kinematic and physiological data at high resolution in unconstrained conditions: a technically challenging goal. Here we present an integrated experimental-computational pipeline for measuring and quantifying body part kinematics and muscle activity in freely behaving Drosophila melanogaster . We first present Spotlight , a closed-loop videography system that performs real-time tracking to record untethered flies at high resolution (6 µm/pixel) and high frame rate (330 Hz) while also enabling optical recordings of limb muscle activity via a fluorescent calcium reporter. To analyze these massive datasets without manual image annotation, we introduce PoseForge , a synthetic-data-driven framework that exploits morphologically accurate biomechanical simulations to generate synthetic data, and contrastive self-supervised learning to infer 3D keypoints and dense body-part segmentation from a single camera view. Using resulting 3D kinematic data, we can replay recorded behaviors in a biomechanical digital twin, NeuroMechFly, to infer forces generated and experienced by the fly’s limbs. Finally, we illustrate the capability of our system to optically record muscle activity. We show how the legs’ long-tendon muscles activate upon mechanical vibration, possibly to activate gripping and to maintain a stable posture. Taken together, this workflow enables scalable, high-resolution measurement and modeling of unconstrained, natural behavior.
- 12 November 2024
- Published by Springer Nature in Nature Methods
- Vol. 21 (12) , 2353-2362
NeuroMechFly v2: simulating embodied sensorimotor control in adult Drosophila
Discovering principles underlying the control of animal behavior requires a tight dialogue between experiments and neuromechanical models. Such models have primarily been used to investigate motor control with less emphasis on how the brain and motor systems work together during hierarchical sensorimotor control. NeuroMechFly v2 expands Drosophila neuromechanical modeling by enabling vision, olfaction, ascending motor feedback and complex terrains that can be navigated using leg adhesion. We illustrate its capabilities by constructing biologically inspired controllers that use ascending feedback to perform path integration and head stabilization. After adding vision and olfaction, we train a controller using reinforcement learning to perform a multimodal navigation task. Finally, we illustrate more bio-realistic modeling involving complex odor plume navigation, and fly–fly following using a connectome-constrained visual network. NeuroMechFly can be used to accelerate the discovery of explanatory models of the nervous system and to develop machine learning-based controllers for autonomous artificial agents and robots.
- 12 November 2024
- Published by Springer Nature in Nature Methods
- Vol. 21 (12) , 2353-2362
NeuroMechFly v2: simulating embodied sensorimotor control in adult Drosophila
Discovering principles underlying the control of animal behavior requires a tight dialogue between experiments and neuromechanical models. Such models have primarily been used to investigate motor control with less emphasis on how the brain and motor systems work together during hierarchical sensorimotor control. NeuroMechFly v2 expands Drosophila neuromechanical modeling by enabling vision, olfaction, ascending motor feedback and complex terrains that can be navigated using leg adhesion. We illustrate its capabilities by constructing biologically inspired controllers that use ascending feedback to perform path integration and head stabilization. After adding vision and olfaction, we train a controller using reinforcement learning to perform a multimodal navigation task. Finally, we illustrate more bio-realistic modeling involving complex odor plume navigation, and fly–fly following using a connectome-constrained visual network. NeuroMechFly can be used to accelerate the discovery of explanatory models of the nervous system and to develop machine learning-based controllers for autonomous artificial agents and robots.
- Open Access
NeuroMechFly v2, simulating embodied sensorimotor control in adult Drosophila
Discovering principles underlying the control of animal behavior requires a tight dialogue between experiments and neuromechanical models. Until now, such models, including NeuroMechFly for the adult fly, Drosophila melanogaster , have primarily been used to investigate motor control. Far less studied with realistic body models is how the brain and motor systems work together to perform hierarchical sensorimotor control. Here we present NeuroMechFly v2, a framework that expands Drosophila neuromechanical modeling by enabling visual and olfactory sensing, ascending motor feedback, and complex terrains that can be navigated using leg adhesion. We illustrate its capabilities by first constructing biologically inspired locomotor controllers that use ascending motor feedback to perform path integration and head stabilization. Then, we add visual and olfactory sensing to this controller and train it using reinforcement learning to perform a multimodal navigation task in closed loop. Finally, we illustrate more biorealistic modeling in two ways: our model navigates a complex odor plume using a Drosophila odor taxis strategy, and it uses a connectome-constrained visual system network to follow another simulated fly. With this framework, NeuroMechFly can be used to accelerate the discovery of explanatory models of the nervous system and to develop machine learning-based controllers for autonomous artificial agents and robots.
Open AccessNeuroMechFly v2, simulating embodied sensorimotor control in adult Drosophila
Discovering principles underlying the control of animal behavior requires a tight dialogue between experiments and neuromechanical models. Until now, such models, including NeuroMechFly for the adult fly, Drosophila melanogaster , have primarily been used to investigate motor control. Far less studied with realistic body models is how the brain and motor systems work together to perform hierarchical sensorimotor control. Here we present NeuroMechFly v2, a framework that expands Drosophila neuromechanical modeling by enabling visual and olfactory sensing, ascending motor feedback, and complex terrains that can be navigated using leg adhesion. We illustrate its capabilities by first constructing biologically inspired locomotor controllers that use ascending motor feedback to perform path integration and head stabilization. Then, we add visual and olfactory sensing to this controller and train it using reinforcement learning to perform a multimodal navigation task in closed loop. Finally, we illustrate more biorealistic modeling in two ways: our model navigates a complex odor plume using a Drosophila odor taxis strategy, and it uses a connectome-constrained visual system network to follow another simulated fly. With this framework, NeuroMechFly can be used to accelerate the discovery of explanatory models of the nervous system and to develop machine learning-based controllers for autonomous artificial agents and robots.