Yuya Shirakawa

This paper presents a time-of-flight image sensor based on 8-Tap P-N junction demodulator (PND) pixels, which is designed for hybrid-type short-pulse (SP)-based ToF measurements under strong ambient light. The 8-tap demodulator implemented with multiple p-n junctions used for modulating the electric potential to transfer photoelectrons to eight charge-sensing nodes and charge drains has an advantage of high-speed demodulation in large photosensitive areas. The ToF image sensor implemented using 0.11 µm CIS technology, consisting of an 120 (H) × 60 (V) image array of the 8-tap PND pixels, successfully works with eight consecutive time-gating windows with the gating width of 10 ns and demonstrates for the first time that long-range (>10 m) ToF measurements under high ambient light are realized using single-frame signals only, which is essential for motion-artifact-free ToF measurements. This paper also presents an improved depth-adaptive time-gating-number assignment (DATA) technique for extending the depth range while having ambient-light canceling capability and a nonlinearity error correction technique. By applying these techniques to the implemented image sensor chip, hybrid-type single-frame ToF measurements with depth precision of maximally 16.4 cm (1.4% of the maximum range) and the maximum non-linearity error of 0.6% for the full-scale depth range of 1.0–11.5 m and operations under direct-sunlight-level ambient light (80 klux) have been realized. The depth linearity achieved in this work is 2.5 times better than that of the state-of-the-art 4-tap hybrid-type ToF image sensor.

This chapter presents semiconductor imaging devices using time-resolved pixel architectures that have been designed for use in biomedical applications. A complementary metal–oxide–semiconductor (CMOS) technology provides an opportunity to implement two types of time-resolved semiconductor-based imaging systems. These are a time-correlated single-photon counting (TCSPC) system, which consists of a single-photon avalanche diode (SPAD) detector, and a time-gating-based time-resolved imaging system, which uses lock-in pixels based on a CMOS image sensor technology. In general, TCSPC-based systems exhibit high accuracies and high time resolution at the cost of complicated logic circuits that are required for almost every pixel to measure the photon-arrival times directly and simultaneously. 234In contrast, time-gating-based systems are characterized by high spatial resolution with relatively simple lock-in pixels. The obtained time-resolution can be very high upon utilization of multiple time-gating windows with multi-tap lock-in pixels and usage of demodulators designed for fast carrier-response. This chapter focuses on time-resolved lock-in-pixel CMOS image sensors for time-gating-based imaging systems and discusses the principle and design of multi-tap lock-in pixels and biomedical applications using time-resolved CMOS image sensors for fluorescence-lifetime imaging microscopy (FLIM), time-resolved near-infrared spectroscopy (NIRS), time-of-flight (TOF)-based spectroscopy, and stimulated Raman spectroscopy (SRS).

An 8-tap CMOS lock-in pixel image sensor that has seven carrier-capturing and a draining time window was developed for short-pulse time-of-flight (TOF) measurements. The proposed pixel for the short-pulse TOF measurements has seven consecutive time-gating windows, each of which has the width of 6 ns, which is advantageous for high-resolution range imaging, particularly for relatively longer distances (>5 m) and under high ambient light operations. In order to enhance the depth resolution, a technique for the depth-adaptive time-gating-number assignment (DATA) for the short-pulse TOF measurement is proposed. A prototype of the 8-tap CMOS lock-in pixel image sensor is implemented with a 1POLY 4METAL 0.11-μm CIS process. The maximum non-linearity error of 1.56%FS for the range of 1–6.4 m and the depth resolution of 6.4 mm was obtained at 6.2 m using the DATA technique.

This paper presents a time-resolved (TR) CMOS image sensor (CIS) using two-tap near-infrared (NIR) lock-in pixels with background light (BGL) cancelling capability for remote heart rate (HR) detection. Aimed to the application, the sensor features a 1024 (V) x 1280 (H) spatial resolution with 412 (V)x 1280 (H) region-of-interest-readout (ROI-readout) at 30 fps, a high sensitivity of 134.8 ke⁻/lux · s, and an ultra-low temporal random noise of 1.13 erms⁻ at analog gain of 64. A maximum charge modulation ratio of 90.1% is obtained for realizing lock-in operation by means of in-pixel lateral electric field charge modulator (LEFM). The remote HR detection is achieved by capturing the heart-beat-induced temporal variation of the oxy-hemoglobin (HbO₂) concentration by the developed TR CIS with NIR lock-in technique, and larger than 98% detection accuracies are attained under both non-visible-light condition and fluctuant brightness ambient-light environments. This remarkable feature demonstrates the superiority of the proposed method for driver monitoring applications in particular.

A programmable sub-nanosecond time-gated four-tap CMOS lock-in pixel (LIP) image sensor that uses an in-pixel pulse generator (PG) is developed for time-resolved (TR) biomedical imaging applications, such as a fluorescence lifetime imaging microscopy (FLIM). We demonstrate the system's effectiveness with simulations and measurements. Using pixel-level PG circuits, very narrow time windows (TWs) that are less than 0.8 ns can be used for charge modulation. This rapid gating allows very weak signals from living cells to be captured with a high signal-to-noise ratio (SNR). The prototype CMOS imaging system displays a fast intrinsic response of 170 ps at 472 nm and a very low temporal random noise of 0.85 e-rms at 45 frames/s under truly correlated-double-sampling operation using two-stage charge transfer. Four outputs from the LIPs allow real-time fluorescence lifetimes (FLTs) measurement, even from samples with multiple FLT components.