Event Spectroscopy: Event-Based Multispectral and Depth Sensing Using Structured Light
- 13 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (3) , 2658-2665
Abstract
Uncrewed aerial vehicles (UAVs) are increasingly deployed in forest environments for tasks such as environmental monitoring and search and rescue, which require safe navigation through dense foliage and precise data collection. Traditional sensing approaches, including passive multispectral and RGB imaging, suffer from latency, poor depth resolution, and strong dependence on ambient light—especially under forest canopies. In this work, we present a novel event spectroscopy system that simultaneously enables high-resolution, low-latency depth reconstruction with integrated multispectral imaging using a single sensor. Depth is reconstructed using structured light, and by modulating the wavelength of the projected structured light, our system captures spectral information in controlled bands between 650 nm and 850 nm. We demonstrate up to 60% improvement in RMSE over commercial depth sensors and validate the spectral accuracy against a reference spectrometer and commercial multispectral cameras, demonstrating comparable performance. A portable version limited to RGB is used to collect real-world depth and spectral data from a Masoala Rainforest. We demonstrate color image reconstruction and material differentiation between leaves and branches using this spectral and depth data. Our results show that adding depth (available at no extra effort with our setup) to material differentiation improves the accuracy by over 30% compared to color-only method. Our system, tested in both lab and real-world rainforest environments, shows strong performance in depth estimation, RGB reconstruction, and material differentiation—paving the way for lightweight, integrated, and robust UAV perception and data collection in complex natural environments.Keywords
All Related Versions
Funding Information
- Swiss National Science Foundation through Eccellenza (186865)
- NCCR Catalysis
This publication has 41 references indexed in Scilit:
- eProbe: Sampling of Environmental DNA within Tree Canopies with DronesEnvironmental Science & Technology, 2024
- Time-Efficient Light-Field Acquisition Using Coded Aperture and Events2024
- Synergistic morphology and feedback control for traversal of unknown compliant obstacles with aerial robotsNature Communications, 2024
- The utility of airborne hyperspectral and satellite multispectral images in identifying Natura 2000 non-forest habitats for conservation purposesScientific Reports, 2023
- Bistable Helical Origami Gripper for Sensor Placement on BranchesAdvanced Intelligent Systems, 2022
- ESL: Event-based Structured Light2021
- Event Guided Depth Sensing2021
- Tree species classification using UAS-based digital aerial photogrammetry point clouds and multispectral imageries in subtropical natural forestsInternational Journal of Applied Earth Observation and Geoinformation, 2020
- EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-TimeInternational Journal of Computer Vision, 2017
- Microsoft COCO: Common Objects in Context2014