M3-RGB: An Imaging Sensor System Using Multicore, Multimode Optical Fiber and Neural Networks
Open Access
Abstract
Conventional image acquisition requires an electrically powered image sensor to be placed directly behind the camera lens, constraining camera placement. To overcome this issue, we introduce M3-RGB as an incoherent-light fiber imaging system in which a multicore, multimode optical fiber passively relays lens images to a remotely located image sensor. Unlike conventional approaches, M3-RGB is designed to operate directly on incoherent light and requires no electrical power or active components at the sensing interface. Because propagation through the fiber yields spatially scrambled patterns, a neural network is used to reconstruct the original scene by exploiting the spatial locality preserved by the multicore structure. In a controlled optical bench setup, where a liquid crystal display monitor displays road-scene images, we construct a paired dataset of scrambled and ground-truth images and quantitatively evaluate reconstruction performance across different fiber core counts, fiber lengths, and calibration settings, utilizing the peak signal-to-noise ratio and structural similarity index measure as performance metrics. By decoupling imaging electronics from the sensing point, this passive remote image relay approach may expand sensor placement options for potential applications such as all-around perception for mobile robots and autonomous vehicles, surveillance, and inspection in confined spaces. Evaluations in real outdoor environments constitute future work.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Varifocal Alvarez metalens array for adaptive light-field imagingNature Communications, 2026
- Intensity-asymmetric wavefront shaping in nonlocal meta-lensNature Communications, 2026
- Streamlined Direct Image Transmission via Plastic Optical Fiber for Automotive ApplicationsIEEE Access, 2025
- Single-ended recovery of optical fiber transmission matrices using neural networksCommunications Physics, 2023
- Unsupervised full-color cellular image reconstruction through disordered optical fiberLight: Science & Applications, 2023
- All-fiber high-speed image detection enabled by deep learningNature Communications, 2022
- nuScenes: A Multimodal Dataset for Autonomous Driving2020
- Scalability in Perception for Autonomous Driving: Waymo Open Dataset2020
- Transmission of natural scene images through a multimode fibreNature Communications, 2019
- Project AutoVision: Localization and 3D Scene Perception for an Autonomous Vehicle with a Multi-Camera System2019