A Three-Wafer-Stacked Hybrid 15-MPixel CIS + 1-MPixel EVS With 4.6-GEvent/s Readout, In-Pixel TDC, and On-Chip ISP and ESP Function
- 24 August 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Solid-State Circuits
- Vol. 58 (11) , 2955-2964
Abstract
We introduce a hybrid 4096 $\times$ 3680 CMOS image sensor (CIS) with an embedded 1032 $\times$ 928 event-based vision sensor (EVS). Within a four-by-four cluster of CIS pixels, one color channel is substituted to provide a photocurrent for an EVS pixel. The EVS readout circuit is allocated on a second wafer and wrapped behind the four-by-four pixel cluster. A third wafer is used to integrate all peripheral readout circuitry for the EVS and CIS functions while achieving a compact sensor footprint. The EVS pixel achieves sub-1 Hz noise rates and 3% contrast non-uniformity (CTNU) at a linear nominal contrast threshold (NCT) of 15%. The row and column readout circuitry bypasses elements without events in a fraction of a clock cycle. Furthermore, a skip function is realized that reads only bordering pixels of a connected set of events. This achieves an event rate of up to 4.6 GEvents/s (GEps). The sensor captures in-pixel time stamps through sampling of a gray code timer realizing a time-to-digital conversion (TDC). Image and event signal processors (ISP & ESP) are integrated on-chip. The ESP provides global ambient and global activity monitoring functions. The sensor supports various flicker mitigation approaches such as adjustment of contrast threshold, in-pixel bandpass filter function and region-of-interest (ROI), and deterministic or random subsampling. The hybrid sensor mitigates challenges in synchronization and parallax error compared with a two-sensor solution using a dedicated CIS and dedicated EVS sensor. Furthermore, only one package and lens are required reducing footprint and cost.Keywords
This publication has 6 references indexed in Scilit:
- A 1280×960 Dynamic Vision Sensor with a 4.95-μm Pixel Pitch and Motion Artifact Minimization2020
- Event-Based Vision: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
- 4.6 A 1/2.3inch 20Mpixel 3-layer stacked CMOS Image Sensor with DRAM2017
- Design of an RGBW color VGA rolling and global shutter dynamic and active-pixel vision sensor2015
- A 240 × 180 130 dB 3 µs Latency Global Shutter Spatiotemporal Vision SensorIEEE Journal of Solid-State Circuits, 2014
- An asynchronous time-based image sensor2008