Song Wang

Existing 3D multi-object tracking (MOT) methods often trade efficiency and generalizability for robustness, as they typically rely on complex association metrics derived from multi-modal architectures or class-specific motion priors. Challenging the common belief that greater complexity necessarily leads to stronger robustness, we propose a robust, efficient, and generalizable method for multi-modal 3D MOT, dubbed RegTrack. Inspired by Yang-Mills gauge theory, RegTrack formulates multi-modal 3D MOT as motion-compensated representation learning. Under this analogy, point-cloud object representations are viewed as matter fields, while inter-frame object motions are regarded as local variations. Geometric cues are modeled as gauge fields to adaptively compensate for such variations, and a pretrained image representation space serves as a globally invariant physical law to guide the compensation process. In this way, the resulting motion-compensated point-cloud representations, viewed as observables, are encouraged to remain consistent for the same object across frames while preserving discriminability among different objects. Their pairwise similarities thus provide a simple yet robust association metric. Specifically, RegTrack is built upon a unified tri-cue encoder (UTEnc), which consists of a local-global point cloud encoder (LG-PEnc), a mixture-of-experts-based geometry encoder (MoE-GEnc), and a frozen image encoder derived from a pretrained vision-language model. LG-PEnc efficiently encodes the spatial-structural information of object point clouds to generate foundational representations. MoE-GEnc interacts with LG-PEnc to model inter-frame geometric relationships and adaptively compensate for motion-induced representation variations without relying on class-specific priors. The frozen image encoder is used only during training to provide a stable representation space for supervising the compensation process, and is discarded during inference. As a result, RegTrack achieves robust, efficient, and generalizable inference using only point-cloud inputs, with merely 2.67 M parameters. Extensive experiments on KITTI and nuScenes demonstrate that RegTrack outperforms its thirty-five competitors.

Person re-identification (Re-ID) is a classical computer vision task and has significant applications for public security and information forensics. Recently, long-term Re-ID with clothes-changing has attracted increasing attention. However, existing methods mainly focus on image-based setting, where richer temporal information is overlooked. In this paper, we focus on the relatively new yet practical problem of Clothes-Changing Video-based Re-ID (CCVReID), which is less studied. First, given the dataset shortage, we build two new benchmark datasets for CCVReID problem, including a large-scale synthetic video dataset and a real-world one, both containing human sequences with various clothing changes. Moreover, we systematically study this problem by simultaneously considering the classical appearance feature and temporal feature contained in the video. We develop a dual-branch fusion framework that makes use of the information from both clothes-aware appearance feature and clothes-free gait feature. For better information fusion, a confidence-guided re-ranking strategy is proposed to adaptively balance the weight of these two categories of features. We have released the benchmark and code proposed in this work to the public at https://github.com/kkw98/CCVReID.

Cross-modal 2D-3D point cloud semantic segmentation using few-shot-based learning provides a practical approach for borrowing matured 2D domain knowledge into the 3D segmentation model, which reduces the reliance on laborious 3D annotation work and improves generalization to new categories. However, previous methods use single-view point cloud generation algorithms to bridge the gap between 2D images and 3D point clouds, leaving the incomplete geometry of an object or scene due to occlusions. To address this issue, we propose a novel view synthesis cross-modal few-shot point cloud semantic segmentation network. It introduces the color and depth inpainting to generate multi-view images and masks, which compensate for the absent depth information of generated point clouds. Additionally, we propose a Co-embedding Network to bridge the domain features between synthesized and original, collected 3D data, and a weighted prototype network is employed to balance the impact of multi-view images and enhance the segmentation performance. Extensive experiments on two benchmarks show the superiority of our method by outperforming the existing cross-modal few-shot 3D segmentation methods.

Face recognition is a widely used computer vision, which plays an increasingly important role in user authentication systems, security systems, and consumer electronics. The models for most current applications are based on high-definition digital cameras. In this paper, we focus on digital images derived from historical motion picture films. Historical motion picture films often have poorer resolution than modern digital imagery, making face detection a more challenging task. To approach this problem, we first propose a trunk–branch concatenated multi-task cascaded convolutional neural network (TB-MTCNN), which efficiently extracts facial features from blurry historical films by combining the trunk with branch networks and employing various sizes of kernels to enrich the multi-scale receptive field. Next, we build a deep neural network-integrated object-tracking algorithm to compensate for failed recognition over one or more video frames. The framework combines simple online and real-time tracking with deep data association (Deep SORT), and TB-MTCNN with the residual neural network (ResNet) model. Finally, a state-of-the-art image restoration method is employed to reduce the effect of noise and blurriness. The experimental results show that our proposed joint face recognition and tracking network can significantly reduce missed recognition in historical motion picture film frames.

Multi-view multi-human association and tracking (MvMHAT), is an emerging yet important problem for multi-person scene video surveillance, aiming to track a group of people over time in each view, as well as to identify the same person across different views at the same time, which is different from previous MOT and multi-camera MOT tasks only considering the over-time human tracking. This way, the videos for MvMHAT require more complex annotations while containing more information for self-learning. In this work, we tackle this problem with an end-to-end neural network in a self-supervised learning manner. Specifically, we propose to take advantage of the spatial-temporal self-consistency rationale by considering three properties of reflexivity, symmetry, and transitivity. Besides the reflexivity property that naturally holds, we design the self-supervised learning losses based on the properties of symmetry and transitivity, for both appearance feature learning and assignment matrix optimization, to associate multiple humans over time and across views. Furthermore, to promote the research on MvMHAT, we build two new large-scale benchmarks for the network training and testing of different algorithms. Extensive experiments on the proposed benchmarks verify the effectiveness of our method. We have released the benchmark and code to the public.