JZJing Zhao

Jing Zhao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Jing Zhao has a total of 26 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Color Spike Camera Reconstruction via Long Short-Term Temporal Aggregation of Spike Signals

    by Yanchen Dong, Ruiqin Xiong, Jing Zhao, Xiaopeng Fan, Xinfeng Zhang, Tiejun Huang

    With the prevalence of emerging computer vision applications, the demand for capturing dynamic scenes with high-speed motion has increased. A kind of neuromorphic sensor called spike camera shows great potential in this aspect since it generates a stream of binary spikes to describe the dynamic light intensity with a very high temporal resolution. Color spike camera (CSC) was recently invented to capture the color information of dynamic scenes via a color filter array (CFA) on the sensor. This paper proposes a long short-term temporal aggregation strategy of spike signals. First, we utilize short-term temporal correlation to adaptively extract temporal features of each time point. Then we align the features and aggregate them to exploit long-term temporal correlation, suppressing undesired motion blur. To implement the strategy, we design a CSC reconstruction network. Based on adaptive short-term temporal aggregation, we propose a spike representation module to extract temporal features of each color channel, leveraging multiple temporal scales. Considering the long-term temporal correlation, we develop an alignment module to align the temporal features. In particular, we perform motion alignment of red and blue channels with the guidance of the higher-sampling-rate green channel, leveraging motion consistency among color channels. Besides, we propose a module to aggregate the aligned temporal features for the restored color image, which exploits color channel correlation. We have also developed a CSC simulator for data generation. Experimental results demonstrate that our method can restore color images with fine texture details, achieving state-of-the-art CSC reconstruction performance.

  • High Spatio-Temporal Imaging Reconstruction for Hybrid Spike-RGB Cameras

    by Lujie Xia, Ruiqin Xiong, Jing Zhao, Lizhi Wang, Shuyuan Zhu, Xiaopeng Fan, Tiejun Huang

    The acquisition of high-resolution image sequence for dynamic scenes of fast motion remains challenging due to motion blur caused by fast object movement. As a novel neuromorphic sensor, spike camera records the changing light intensity via spike stream of ultra-high temporal resolution, excelling in motion recording but limited in spatial resolution. This paper proposes a method for high spatio-temporal resolution (HSTR) imaging with a hybrid Spike-RGB camera, utilizing the information from spike stream to enhance the temporal resolution and the information from RGB images to enhance the spatial resolution of texture details. For this purpose, we present HSTR-Net, a dedicated network to process the spike and RGB data, which incorporates three key innovations: 1) A temporal control encoder enabling flexible temporal reconstruction through spike stream processing with embedded time parameters, eliminating the requirement to train multiple inference models; 2) Motion-aware feature projection that aligns RGB frame details to target timestamps using spike-derived motion offsets; 3) An adaptive transformer-based fusion strategy establishing cross-modal spatial correlations through mutual attention mechanisms. Extensive experiments demonstrate state-of-the-art performance on synthetic benchmark datasets with 5.23 dB PSNR and 6.94% SSIM improvement. It also shows visually impressive performance on real-world captured spike dataset.


  • High Dynamic Range Imaging for Dynamic Scenes Based on Multi-Level Spike Camera

    by Zhenkun Zhu, Ruiqin Xiong, Jing Zhao, Rui Zhao, Xiaopeng Fan, Shuyuan Zhu, Tiejun Huang

    Spike camera is a retina-inspired neuromorphic camera which can capture dynamic scenes of high-speed motion by firing a continuous stream of spikes at an extremely high temporal resolution. The limitation in the current design is that each spike only represents the arrival of a fixed amount of photons. It can not deal with strong light areas in which the amount of accumulated photons reaches the pre-specified threshold multiple times within a single readout interval. In this paper, we propose a new spike camera model of high-speed imaging for high dynamic range scenarios. In this scheme, each pixel accumulates the incoming photons persistently and generates a new type of spike stream in which each spike symbol may be associated with different levels, indicating the arrival of different amounts of photons since the last readout. This enables the camera to support dynamic scenes with wider dynamic range. To achieve this, we propose a two-level buffer mechanism, one for photon accumulation and one for spike-firing encoding. We use a register to hold the number of spike-firings which has not been read out yet. At each readout time, the major part in the counter is read out via a carefully designed exponential encoding and the counter is updated. Such encoding and readout strategy enables a very efficient expansion of the dynamic range using a small number of encoding bits. Furthermore, we propose an image reconstruction scheme for the proposed camera, utilizing both spike intervals and spike levels to recover the light intensity. We incorporate Mamba and propose a temporal-spatial selective scan mechanism to extract temporal-spatial correlation within spike streams. We employ a pyramid adaptive filtering and alignment module to achieve coarse-to-fine feature alignment. Experimental results show that the proposed scheme can achieve better imaging quality and outperform the existing spike camera in high dynamic range scenarios.


  • Learning a Deep Demosaicing Network for Spike Camera With Color Filter Array

    by Yanchen Dong, Ruiqin Xiong, Jing Zhao, Jian Zhang, Xiaopeng Fan, Shuyuan Zhu, Tiejun Huang

    For capturing dynamic scenes with ultra-fast motion, neuromorphic cameras with extremely high temporal resolution have demonstrated their great capability and potential. Different from the event cameras that only record relative changes in light intensity, spike camera fires a stream of spikes according to a full-time accumulation of photons so that it can recover the texture details for both static areas and dynamic areas. Recently, color spike camera has been invented to record color information of dynamic scenes using a color filter array (CFA). However, demosaicing for color spike cameras is an open and challenging problem. In this paper, we develop a demosaicing network, called CSpkNet, to reconstruct dynamic color visual signals from the spike stream captured by the color spike camera. Firstly, we develop a light inference module to convert binary spike streams to intensity estimates. In particular, a feature-based channel attention module is proposed to reduce the noises caused by quantization errors. Secondly, considering both the Bayer configuration and object motion, we propose a motion-guided filtering module to estimate the missing pixels of each color channel, without undesired motion blur. Finally, we design a refinement module to improve the intensity and details, utilizing the color correlation. Experimental results demonstrate that CSpkNet can reconstruct color images from the Bayer-pattern spike stream with promising visual quality.


  • Open Access

    A Universal Optimization Framework for Learning-based Image Codec

    by Jing Zhao, Bin Li, Jiahao Li, Ruiqin Xiong, Yan Lu

    Recently, machine learning-based image compression has attracted increasing interests and is approaching the state-of-the-art compression ratio. But unlike traditional codec, it lacks a universal optimization method to seek efficient representation for different images. In this paper, we develop a plug-and-play optimization framework for seeking higher compression ratio, which can be flexibly applied to existing and potential future compression networks. To make the latent representation more efficient, we propose a novel latent optimization algorithm to adaptively remove the redundancy for each image. Additionally, inspired by the potential of side information for traditional codecs, we introduce side information into our framework, and integrate side information optimization with latent optimization to further enhance the compression ratio. In particular, with the joint side information and latent optimization, we can achieve fine rate control using only single model instead of training different models for different rate-distortion trade-offs, which significantly reduces the training and storage cost to support multiple bit rates. Experimental results demonstrate that our proposed framework can remarkably boost the machine learning-based compression ratio, achieving more than 10% additional bit rate saving on three different representative network structures. With the proposed optimization framework, we can achieve 7.6% bit rate saving against the latest traditional coding standard VVC on Kodak dataset, yielding the state-of-the-art compression ratio.


  • Construction of Smart Pension System for Urban and Rural Residents Based on Big Data Mining Algorithm

    by Jing Zhao

    Population aging is a social problem that every country cannot avoid in the process of modernization. China has rapidly stepped into the threshold of an aging society. With the deepening of the aging of the population and with the increasing demand for diversified services for the elderly, there is an urgent need to provide multi-level, high-quality elderly care(ec) services for the elderly. There is a large gap in human resources and other resources in the traditional pension industry. The pension industry needs to develop. Zhiyi pension has opened a new realm of the pension model. It cannot only provide a variety of life assistance, Rehabilitation care, psychological comfort and social support services for the elderly through intelligent decision-making and control, but also always protect the safety of the elderly and meet the multi-level and diversified service The needs of urban and rural elderly. We should play the leading comprehensive role of innovation driving role of information technology, accelerate the development of smart health care industry, improve people's livelihood and cultivate new economic drivers, On the 23rd of 2017, the Ministry of Industry, Information Technology and Information Technology, the Ministry of Civil Affairs and the Health and Family Planning Commission jointly issued the Action Plan for the Development of Smart ec Industry (2017-2022), which explicitly mentioned the year 2022, It will "basically form a smart health care industry system covering the whole life cycle. So far, the Smart Pension (SM) industry has completed its overall layout and entered the practical level.

  • Reconstructing Clear Image for High-Speed Motion Scene With a Retina-Inspired Spike Camera

    by Jing Zhao, Ruiqin Xiong, Jiyu Xie, Boxin Shi, Zhaofei Yu, Wen Gao, Tiejun Huang

    Conventional digital cameras typically accumulate all the photons within an exposure period to form a snapshot image. It requires the scene to be quite still during the imaging time, otherwise it would result in blurry image for the moving objects. Recently, a retina-inspired spike camera has been proposed and shown great potential for recording high-speed motion scenes. Instead of capturing the visual scene by a single snapshot, the spike camera records the dynamic light intensity variation continuously. Each pixel on spike camera sensor accumulates the incoming photons independently and persistently, which fires a spike and restarts the photon accumulation immediately once the dispatch threshold is reached, producing a continuous stream of spikes recorded at very high temporal resolution. To recover the dynamic scene from captured spike stream, this paper presents an image reconstruction approach for spike camera. In order to generate high-quality reconstruction, we investigate the temporal correlation along motion trajectories and exploit it via adaptive temporal filtering. In particular, we present a hierarchical motion-aligned temporal filtering scheme, combining short-term filtering with long-term filtering to take advantage of long-term temporal correlation with low model complexity. Experimental results demonstrate that the proposed scheme outperforms the existing schemes significantly, producing much better objective and subjective qualities for spike camera image reconstruction.


  • Place recognition with deep superpixel features for brain-inspired navigation

    by Jing Zhao, Jun Tang, Donghua Zhao, Huiliang Cao, Xiaochen Liu, Chong Shen, Chenguang Wang, Jun Liu

    Navigation in primates is generally supported by cognitive maps. Such a map endows an animal with navigational planning capabilities. Numerous methods have been proposed to mimic these natural navigation capabilities in artificial systems. Based on self-navigation and learning strategies in animals, we propose in this work a place recognition strategy for brain-inspired navigation. First, a place recognition algorithm structure based on convolutional neural networks (CNNs) is introduced, which can be applied in the field of intelligent navigation. Second, sufficient images are captured at each landmark and then stored as a reference image library. Simple linear iterative clustering (SLIC) is used to segment each image into superpixels with multi-scale viewpoint-invariant landmarks. Third, highly representative appearance-independent features are extracted from these landmarks through CNNs. In addition, spatial pyramid pooling (SPP) layers are introduced to generate a fixed-length CNN representation, regardless of the image size. This representation boosts the quality of the extracted landmark features. The proposed SLIC-SPP-CNN place recognition algorithm is evaluated on one collected dataset and two public datasets with viewpoint and appearance variations.


  • Residual Signals Modeling for Layered Image/Video Softcast with Hybrid Digital-Analog Transmission

    by Jing Zhao, Jiyu Xie, Ruiqin Xiong

    Recently, the SoftCast scheme has shown great potential for robust image/video communication in wireless scenarios, where the channel quality may fluctuate drastically and unpredictably. However, the analog-like transmission in Soft-Cast is not always efficient in terms of power usage, compared with digital approaches. In this paper, we propose a layered image/video SoftCast scheme, in which a coarse version of the image is transmitted by a base layer in digital way, while the residual details are delivered by an enhancement layer in pseudo-analog way. In order to achieve optimal overall transmission performance, this paper studies the problem of optimal bit rate selection for the base layer, by building a rate-residual model based on the relationship between the base layer bit rate and the residual signal spectrum. Experimental results show that the proposed scheme can improve the performance of original scheme remarkably, while still preserving the smooth quality degradation characteristic of SoftCast.