Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Shenyong Zhang has a total of 14 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Extreme R-CNN: Few-Shot Object Detection via Sample Synthesis and Knowledge Distillation

    by Shenyong Zhang, Wenmin Wang, Zhibing Wang, Honglei Li, Ruochen Li, Shixiong Zhang

    Traditional object detectors require extensive instance-level annotations for training. Conversely, few-shot object detectors, which are generally fine-tuned using limited data from unknown classes, tend to show biases toward base categories and are susceptible to variations within these unknown samples. To mitigate these challenges, we introduce a Two-Stage Fine-Tuning Approach (TFA) named Extreme R-CNN, designed to operate effectively with extremely limited original samples through the integration of sample synthesis and knowledge distillation. Our approach involves synthesizing new training examples via instance clipping and employing various data-augmentation techniques. We enhance the Faster R-CNN architecture by decoupling the regression and classification components of the Region of Interest (RoI), allowing synthetic samples to train the classification head independently of the object-localization process. Comprehensive evaluations on the Microsoft COCO and PASCAL VOC datasets demonstrate significant improvements over baseline methods. Specifically, on the PASCAL VOC dataset, the average precision for novel categories is enhanced by up to 15 percent, while on the more complex Microsoft COCO benchmark it is enhanced by up to 6.1 percent. Remarkably, in the 1-shot scenario, the AP50 of our model exceeds that of the baseline model in the 10-shot setting within the PASCAL VOC dataset, confirming the efficacy of our proposed method.


  • Open Access

    Recovery-Based Occluded Face Recognition by Identity-Guided Inpainting

    by Honglei Li, Yifan Zhang, Wenmin Wang, Shenyong Zhang, Shixiong Zhang

    Occlusion in facial photos poses a significant challenge for machine detection and recognition. Consequently, occluded face recognition for camera-captured images has emerged as a prominent and widely discussed topic in computer vision. The present standard face recognition methods have achieved remarkable performance in unoccluded face recognition but performed poorly when directly applied to occluded face datasets. The main reason lies in the absence of identity cues caused by occlusions. Therefore, a direct idea of recovering the occluded areas through an inpainting model has been proposed. However, existing inpainting models based on an encoder-decoder structure are limited in preserving inherent identity information. To solve the problem, we propose ID-Inpainter, an identity-guided face inpainting model, which preserves the identity information to the greatest extent through a more accurate identity sampling strategy and a GAN-like fusing network. We conduct recognition experiments on the occluded face photographs from the LFW, CFP-FP, and AgeDB-30 datasets, and the results indicate that our method achieves state-of-the-art performance in identity-preserving inpainting, and dramatically improves the accuracy of normal recognizers in occluded face recognition.


  • Open Access

    Bimodality Cross Concatenation and Fusion for Audio-visual Event Localization

    by Ruochen Li, Wenmin Wang, Wenying Wei, Tongbao Chen, Shenyong Zhang

    Audio-visual event localization (AVEL) is a task that utilizes audio and visual features in videos to perceive the correlation between audio and video and the type of event. Previous methods have mostly focused on aligning the two features in temporal sequence, ignoring the high-order feature expression after audio-visual feature fusion and the role of cross-attention. To address this issue, we propose a bimodal feature cross-concatenation fusion network (BiCCF Net) that aligns visual and audio features in latent space using the spatiotemporal correlation (STC) module. And the audio-visual cross attention (AVCA) module is used to extract cross-attention while using the Factorized Bilinear Coding (FBC) based Audio-Visual Fusion (AVF) Module to obtain the fused high-order feature expression. Finally, the fused features are combined with cross-attention and processed by a background suppression classification module to predict the category of events and the correlation between audio and video features. Our experiments were conducted on the AVE dataset, and we achieved significant improvement compared to baseline models.

  • Open Access

    E-detector: Asynchronous Spatio-temporal for Event-based Object Detection in Intelligent Transportation System

    by Shixiong Zhang, Wenmin Wang, Honglei Li, Shenyong Zhang

    In intelligent transportation systems, various sensors, including radar and conventional frame cameras, are used to improve system robustness in various challenging scenarios. An event camera is a novel bio-inspired sensor that has attracted the interest of several researchers. It provides a form of neuromorphic vision to capture motion information asynchronously at high speeds. Thus, it possesses advantages for intelligent transportation systems that conventional frame cameras cannot match, such as high temporal resolution, high dynamic range, as well as sparse and minimal motion blur. Therefore, this study proposes an E-detector based on event cameras that asynchronously detect moving objects. The main innovation of our framework is that the spatiotemporal domain of the event camera can be adjusted according to different velocities and scenarios. It overcomes the inherent challenges that traditional cameras face when detecting moving objects in complex environments, such as high speed, complex lighting, and motion blur. Moreover, our approach adopts filter models and transfer learning to improve the performance of event-based object detection. Experiments have shown that our method can detect high-speed moving objects better than conventional cameras using state-of-the-art detection algorithms. Thus, our proposed approach is extremely competitive and extensible, as it can be extended to other scenarios concerning high-speed moving objects. The study findings are expected to unlock the potential of event cameras in intelligent transportation system applications.


  • Bounding convolutional network for refining object locations

    by Shenyong Zhang, Wenmin Wang, Honglei Li, Shixiong Zhang


  • Open Access

    EVtracker: An Event-Driven Spatiotemporal Method for Dynamic Object Tracking

    by Shixiong Zhang, Wenmin Wang, Honglei Li, Shenyong Zhang

    An event camera is a novel bio-inspired sensor that effectively compensates for the shortcomings of current frame cameras, which include high latency, low dynamic range, motion blur, etc. Rather than capturing images at a fixed frame rate, an event camera produces an asynchronous signal by measuring the brightness change of each pixel. Consequently, an appropriate algorithm framework that can handle the unique data types of event-based vision is required. In this paper, we propose a dynamic object tracking framework using an event camera to achieve long-term stable tracking of event objects. One of the key novel features of our approach is to adopt an adaptive strategy that adjusts the spatiotemporal domain of event data. To achieve this, we reconstruct event images from high-speed asynchronous streaming data via online learning. Additionally, we apply the Siamese network to extract features from event data. In contrast to earlier models that only extract hand-crafted features, our method provides powerful feature description and a more flexible reconstruction strategy for event data. We assess our algorithm in three challenging scenarios: 6-DoF (six degrees of freedom), translation, and rotation. Unlike fixed cameras in traditional object tracking tasks, all three tracking scenarios involve the simultaneous violent rotation and shaking of both the camera and objects. Results from extensive experiments suggest that our proposed approach achieves superior accuracy and robustness compared to other state-of-the-art methods. Without reducing time efficiency, our novel method exhibits a 30% increase in accuracy over other recent models. Furthermore, results indicate that event cameras are capable of robust object tracking, which is a task that conventional cameras cannot adequately perform, especially for super-fast motion tracking and challenging lighting situations.


  • Open Access

    Eventmd: High-Speed Moving Object Detection Based on Event-Based Video Frames

    by Shixiong Zhang, Wenmin Wang, Honglei Li, Shenyong Zhang

    The event camera is a novel bio-inspired sensor that has attracted the interest of many researchers. As a neuromorphic vision, the sensor is different from the traditional frame-based cameras. It has such advantages for high-speed object detection that conventional frame-based cameras can’t match, e.g., high temporal resolution, high dynamic range (HDR), sparse and minimal motion blur. In this paper, we propose EventMD an adaptive framework for high-speed moving object detection using event cameras and an event-based dataset collected from real traffic scenes using event-camera Celex-V. The key novelties of our framework are our method overcomes the inability of conventional cameras to detect high-speed moving objects and also provides a framework for event vision that can adapt to any detector. Since event cameras have unique advantages for capturing high-speed moving objects in different scenes, our framework combines bio-inspired sensors and conventional detection algorithms to detect high-speed moving objects and to reduce motion blur and data redundancy. The experiments show that our method can detect high-speed moving objects but some state-of-the-art detection algorithms with the conventional camera are not capable of it and our proposed approach is very competitive and extensible. In addition, our approach can be extended to other scenarios of high-speed moving objects.


  • BAYESIAN NETWORK REVISION WITH PROBABILISTIC CONSTRAINTS

    by Yun Peng, Zhongli Ding, Shenyong Zhang, Rong Pan

    This paper deals with an important probabilistic knowledge integration problem: revising a Bayesian network (BN) to satisfy a set of probability constraints representing new or more specific knowledge. We propose to solve this problem by adopting IPFP (iterative proportional fitting procedure) to BN. The resulting algorithm E-IPFP integrates the constraints by only changing the conditional probability tables (CPT) of the given BN while preserving the network structure; and the probability distribution of the revised BN is as close as possible to that of the original BN. Two variations of E-IPFP are also proposed: 1) E-IPFP-SMOOTH which deals with the situation where the probabilistic constraints are inconsistent with each other or with the network structure of the given BN; and 2) D-IPFP which reduces the computational cost by decomposing a global E-IPFP into a set of smaller local E-IPFP problems.