Zhou Ge

Neuromorphic imaging sensors (event cameras) offer a promising paradigm for computational imaging and human pose estimation (HPE) under extreme illumination conditions. Nevertheless, dark-scene background activity originating from photodiode dark current and circuit thermal noise, together with hot-pixel noise, severely corrupts event streams and impedes reliable HPE in low-light scenarios. To address this issue, we propose an adaptive event denoising framework built upon a spatiotemporal Gaussian-weighted neighborhood model with a dynamic thresholding mechanism. It can effectively suppress background activity and hot-pixel noise while preserving edge and motion details critical for pose estimation. Leveraging this denoising front-end, we construct a complete dark-scene neuromorphic HPE pipeline by transferring the pre-trained MediaPipe model onto event-based time-surfaces. Quantitative and qualitative evaluations on public and self-collected datasets demonstrate that our approach outperforms state-of-the-art denoising methods with an improvement of over 20% on public benchmarks and over 30% on self-collected dark-scene data. We expect our work to pave the way toward reliable dark-scene human–robot interaction through robust neuromorphic pose estimation.

Digital holography is a powerful technique for the four-dimensional (4D) spatio-temporal reconstruction of microparticles in microfluidic systems, which is crucial for applications in fluid dynamics and biology. However, the accuracy of conventional in-line holography is often hindered by significant background noise and artifacts. To overcome these limitations, we propose what is believed to be a novel compressive dark-field digital holography (CD-DH) method. This approach integrates a dark-field operation to enhance the signal-to-noise ratio (SNR) by suppressing background interference, and leverages compressed sensing (CS) with a total variation (TV) regularization constraint to computationally eliminate the twin image problem, enabling high-precision 4D reconstruction from a sequence of consecutively recorded holograms. We systematically validated the performance of our method through simulations and experiments. In simulations, CD-DH reduced the mean absolute error (MAE) and root mean square error (RMSE) of particle localization by approximately 67.9% and 67.5%, respectively, compared to the conventional back-propagation (BP) algorithm. Experimentally, when analyzing polystyrene microspheres in a uniform flow, our method achieved a velocity measurement error of 2%, a stark improvement over the error from the BP method. Furthermore, in a non-uniform flow within a Y-shaped microfluidic chip, our technique effectively removed background artifacts and accurately reconstructed particle trajectories and velocity fields that align well with Poiseuille flow theory. These results demonstrate that CD-DH is a robust and precise tool, providing powerful technical support for the quantitative visualization of complex micro-scale flow phenomena.

Digital holography has emerged as a powerful tool for investigating dynamic particle behaviors in the spatiotemporal domain. However, the performance of this technique is fundamentally limited by the system’s space–time bandwidth, leading to a trade-off between accuracy and speed. This study presents a novel, to the best of our knowledge, approach integrating event-driven neuromorphic imaging with digital holography for dynamic particle imaging. Through comprehensive analysis of synthetic and real-world experiments, we demonstrate that the proposed method significantly improves data efficiency and hologram contrast by over tenfold. This technique, termed “neuromorphic holography,” offers promising applications for efficient dynamic particle imaging.

We present a design approach for a symmetric power exponential phase, resulting in a phase-modulated point-spread function with two symmetrical mainlobes that can be steered toward the optical axis opposite the x axis. This design offers nanoscale 3D localization capabilities suitable for 3D single-molecule tracking and localization imaging. The axial probing depth of this symmetric power exponential point-spread function can be adjusted as needed by manipulating specific parameters. Optimization of the symmetric power exponential phase involves truncation filtering to reduce sidelobes and the utilization of a phase inversion-based optimization algorithm to enhance transfer function efficiency and localization precision. A successful multi-molecule 3D tracking experiment was conducted at a 10 µm axial depth using a numerical aperture of 1.4 to validate the efficacy of the proposed design methodology.

Non-line-of-sight (NLOS) sensing is an emerging technique that is capable of detecting objects hidden behind a wall, around corners, or behind other obstacles. However, NLOS tracking of moving objects is challenging due to signal redundancy and background interference. Here, we demonstrate computational neuromorphic imaging with an event camera for NLOS tracking, unaffected by the relay surface, which can efficiently obtain non-redundant information. We show how this sensor, which responds to changes in luminance within dynamic speckle fields, allows us to capture the most relevant events for direct motion estimation. The experimental results confirm that our method has superior performance in terms of efficiency, and accuracy, which greatly benefits from focusing on well-defined NLOS object tracking.

Neuromorphic imaging reacts to per-pixel brightness changes of a dynamic scene with high temporal precision and responds with asynchronous streaming events as a result. It also often supports a simultaneous output of an intensity image. Nevertheless, the raw events typically involve a large amount of noise due to the high sensitivity of the sensor, while capturing fast-moving objects at low frame rates results in blurry images. These deficiencies significantly degrade human observation and machine processing. Fortunately, the two information sources are inherently complementary — events with microsecond-level temporal resolution, which are triggered by the edges of objects recorded in a latent sharp image, can supply rich motion details missing from the blurry one. In this work, we bring the two types of data together and introduce a simple yet effective unifying algorithm to jointly reconstruct blur-free images and noise-robust events in an iterative coarse-to-fine fashion. Specifically, an event-regularized prior offers precise high-frequency structures and dynamic features for blind deblurring, while image gradients serve as a kind of faithful supervision in regulating neuromorphic noise removal. Comprehensively evaluated on real and synthetic samples, such a synergy delivers superior reconstruction quality for both images with severe motion blur and raw event streams with a storm of noise, and also exhibits greater robustness to challenging realistic scenarios such as varying levels of illumination, contrast and motion magnitude. Meanwhile, it can be driven by much fewer events and holds a competitive edge at computational time overhead, rendering itself preferable as available computing resources are limited. Our solution gives impetus to the improvement of both sensing data and paves the way for highly accurate neuromorphic reasoning and analysis.

Bio-inspired neuromorphic cameras asynchronously record visual information of dynamic scenes by discrete events. Due to the high sampling rate, they are capable of fast motion capture without causing image blur, overcoming the drawbacks of frame-based cameras that produce blurry recordings of dynamic objects. However, highly sensitive neuromorphic cameras are also susceptible to interference, and can generate a lot of noise in response. Such noisy event data can dramatically degrade the event-based observations and analysis. Existing methods have insufficient performance on noise suppression, especially for the weak dynamic scenes where noise resembles signals in attributes and distribution, and their results thus have limited improvements on downstream applications. Such demanding cases have not been fully investigated. We aim to seek a solution with more effective and robust discrimination between the two types of events, such that the denoised output can benefit neuromorphic classification tasks. Therefore, we propose a sub-quadratic clustering algorithm tailored for neuromorphic data. It couples event priors with density estimation for noise removal on raw event streams, where strongly correlated signals are taken to be denser in space-time. Experiments on real and synthetic samples illustrate that our simple and interpretable algorithm can suppress noise significantly, and can show greater accuracy and robustness than other techniques in some challenging scenarios.

Inverse imaging covers a wide range of imaging applications, including super-resolution, deblurring, and compressive sensing. We propose a novel scheme to solve such problems by combining duality and the alternating direction method of multipliers (ADMM). In addition to a conventional ADMM process, we introduce a second one that solves the dual problem to find the estimated nontrivial lower bound of the objective function, and the related iteration results are used in turn to guide the primal iterations. We call this D-ADMM, and show that it converges to the global minimum when the regularization function is convex and the optimization problem has at least one optimizer. Furthermore, we show how the scheme can give rise to two specific algorithms, called D-ADMM-L2 and D-ADMM-TV, by having different regularization functions. We compare D-ADMM-TV with other methods on image super-resolution and demonstrate comparable or occasionally slightly better quality results. This paves the way of incorporating advanced operators and strategies designed for basic ADMM into the D-ADMM method as well to further improve the performances of those methods.