Hybrid Event–Frame Sensing for Human-Perceptual Imaging and Machine Vision
Open Access
Abstract
Frame-based RGB image sensors and event-based vision sensors provide complementary sensing capabilities for human-perceptual imaging and machine vision. RGB image sensors capture dense spatial, color, and texture information that is essential for human-viewable imaging, semantic recognition, and conventional image signal processing pipelines. In contrast, dynamic vision sensors (DVSs) and event vision sensors (EVSs) asynchronously detect local brightness changes and provide sparse temporal information with low latency, high temporal resolution, and reduced redundant data output. Because neither modality alone satisfies all requirements of emerging vision systems, hybrid event–frame sensing has become an important direction for compact, low-latency, and energy-efficient sensing. This review presents a sensor-oriented taxonomy of hybrid event–frame sensing architectures and systems, including dual-camera event–frame systems, optically aligned event–frame systems, pixel-level shared hybrid image sensors, stacked CIS–DVS hybrid image sensors, homogeneous-pixel sensing systems, and event-only reconstruction systems. We analyze key sensor specifications, including latency, spatial resolution, color fidelity, power consumption, and form factor, and discuss how these specifications guide sensor configuration and design. The review identifies stacked CIS–DVS sensors as one of the most balanced and competitive architectures because they can support compact integration, synchronized event–frame sensing, and on-chip processing. However, important challenges remain, including color fidelity, demosaicing, event-pixel ratio optimization, calibration, benchmarking, and edge-AI deployment. Finally, we emphasize that future hybrid event–frame sensing systems should be developed through sensor–algorithm–ISP–AI co-design. This review provides practical guidelines for developing next-generation hybrid event–frame sensing systems for both human-perceptual imaging and machine vision.Keywords
This publication has 161 references indexed in Scilit:
- EIFNet: Leveraging Event-Image Fusion for Robust Semantic Segmentation2026
- Recent Event Camera Innovations: A Survey2025
- DailyDVS-200: A Comprehensive Benchmark Dataset for Event-Based Action Recognition2024
- DVS-Voltmeter: Stochastic Process-Based Event Simulator for Dynamic Vision Sensors2022
- Simple Baselines for Image Restoration2022
- Event-Based Fusion for Motion Deblurring with Cross-modal Attention2022
- Reducing the Sim-to-Real Gap for Event Cameras2020
- Correction to: PCA-RECT: An Energy-Efficient Object Detection Approach for Event Cameras2019
- Real-Time Intensity-Image Reconstruction for Event Cameras Using Manifold RegularisationInternational Journal of Computer Vision, 2018
- The development of the CIE 2000 colour‐difference formula: CIEDE2000Color Research & Application, 2001