Jun Hu

The online vectorized map construction technique employs a neural network model to forecast the vectorized representation of a specific region around automobiles, using data obtained from sensors mounted on automobiles. Due to advances in end-to-end object detection with transformers framework, the research on query-based online mapping has attracted substantial attention. However, the fixed number of queries and the random initialization of query embeddings constrain the model's performance. Moreover, the transformer architecture for object detection is based on the assumption that queries are identically distributed and independent, a premise that is not entirely applicable to map point queries which possess established subordinate relationships with map element instances. To address these issues, we initially incorporate a simplified transformer layer that utilizes semantic priors in bird's-eye view features for query initialization. The queries are then sent to a transformer-based map decoder for optimization and combined with a dynamic query management mechanism to eliminate low-confidence queries, hence maintaining computational efficiency. Furthermore, to guarantee that point queries within each instance preserve a consistent representation and avoid feature confusion among map element instances, we proposed an instance internal consistency map decoder. We conduct extensive experiments on commonly used map construction datasets to evaluate the proposed method. The experimental results demonstrate that our proposed method achieves state-of-the-art performance on the nuScenes and Argoverse 2 datasets.

Multi-camera 3D object detection receives extensive attention in autonomous driving because of its advantage in efficiency and cost-effectiveness. However, a performance gap still exists between camera-based detector and LiDAR-based detector. Because camera-based detector cannot extract precise depth and geometric information from the input image data, resulting in obvious projection deviation when transforming features from the perspective view to the bird’s eye view (BEV). We propose a cross-modal knowledge distillation framework (DFPR-BEV) to address this problem, making the camera-based student detector to improve its feature representation by imitating the trained LiDAR-based teacher detector during training. To prevent semantic information from being harmed by directly aligning features during distillation by modal variation, we improve the student model to decouple point clouds feature distillation and image projection position correction. A heatmap-based projection correction method is proposed to correct image feature projection deviation and eliminate conflicting features. A Gaussian mixture region-based method is introduced for comprehensive geometry and depth transmission of point clouds features. Finally, the detection performance is improved by effectively fusing the two distilled features. Extensive experiments on nuScenes benchmark illustrate that DFPR-BEV can significantly improve the performance of the student model, outperforming current distillation methods and other camera-based detectors with equal or lower computational budget. Notably, on the nuScenes val set after distillation, BEVDet’s NDS and mAP are improved by 4.9% and 4.2%, respectively, BEVDet4D’s inference speed is 5× faster than BEVFormer-S with the same mAP and higher NDS.

In highly dynamic and complex autonomous driving environments, accurately predicting agents’ future multimodal trajectories still faces challenges such as modeling diverse social interactions, capturing dynamic intents, and ensuring prediction consistency. To address these issues, this paper proposes a novel trajectory prediction model that integrates a Hierarchical Recursive Interaction Network (HRINet) and a multi-stage goal-guided mechanism (GoalNet), aiming to improve prediction accuracy, stability, and plausibility. Specifically, we design a HRINet with local and global attention mechanisms to recursively model various social interactions, while progressively integrating map semantic information to enhance the model’s understanding of traffic scenes. Meanwhile, inspired by the divide-and-conquer approach, the proposed GoalNet first estimates fine-grained multi-stage goal lane segments along the path. These goals are then used to continuously guide and constrain the trajectory generation process, effectively reducing error accumulation and improving stability. In addition, we construct a dynamic goal candidate area that combines domain knowledge and traffic rules to filter out unreasonable goals, thereby enhancing the plausibility and consistency of the predictions. Experimental results on nuScenes, INTERACTION, and Waymo Open Motion Dataset (WOMD) show that our model achieves state-of-the-art performance in multiple key metrics, maintains a trade-off between prediction accuracy, model complexity, and inference latency, and shows high stability and consistency in predictions.

Bird’s Eye View (BEV) perception requires swift and accurate representation of real-world 3D information, and BEV-based 3D object detection and semantic segmentation are foundational for downstream autonomous tasks such as prediction and planning. Current BEV perception methods are divided into Light Detection and Ranging(LiDAR)-based and camera-based methods depending on the input data modalities. LiDAR-based methods offer precise 3D information, but are limited by high sensor costs. Conversely, camera-based methods are economical and widely researched. However, 2D image features transform into BEV space introducing feature distortion due to inaccurate 3D information, particularly in depth estimation which results in noisy projections. Denoising Diffusion Probabilistic Models (DDPM) are used to mitigate noise in BEV features. However, DDPM increases computational costs, relies on noisy images BEV features, and limits effectiveness. To address these issues, we propose a lightweight BEV perception framework (DMPG-BEV) to enhance the performance of camera-based methods. This framework employs Denoising Diffusion Implicit Models (DDIM) to generate point cloud features with 3D information, while reducing computational demands by fewer denoising steps. We also introduce a lightweight feature fusion module which integrates point cloud BEV features with image BEV features to incorporate 3D information and reduce noise. The refined BEV features are also well-suited for multi-task networks involving detection and segmentation tasks. Additionally, a lightweight view transformation module is included to further reduce computational costs. Extensive experiments on the nuScenes benchmark demonstrate better performance of our proposed model with reduced computation, highlighting its value in BEV perception.

Visual Place Recognition (VPR) is crucial for various computer vision and robotics applications. Traditional VPR techniques relying on handcrafted features, have been enhanced by using Convolutional Neural Networks (CNNs). Recently, MixVPR has set new benchmarks in VPR by using advanced feature aggregation techniques. However, MixVPR's full-image feature mixing approach can lead to the ignoring of critical local detail information and regional saliency information in large-scale images. To overcome this, we propose MixVPR++, which integrates an Adaptive Gabor Texture Fuser with a Learnable Gabor Filter for enriching semantic context with texture details information and a Hierarchical-Region Feature-Mixer for better spatial hierarchy capture regional saliency information, thereby enhancing robustness and accuracy. Extensive experiments demonstrate that MixVPR++ outperforms state-of-the-art methods across most challenging benchmarks. Despite its impressive performance, MixVPR++ shows limitations in handling severe viewpoint changes, indicating an area for future improvement.

Visual Place Recognition (VPR) is a fundamental task in robotics and computer vision, where the ability to recognize locations from visual inputs is crucial for autonomous navigation systems. Traditional methods, which rely on handcrafted features or standard convolutional neural networks (CNNs), struggle with environmental changes that significantly alter a place’s appearance. Recent advancements in deep learning have improved VPR by focusing on deep-learned features, enhancing robustness under varying conditions. However, these methods often overlook saliency cues, leading to inefficiencies in dynamic scenes. To address these limitations, we propose an improved MixVPR model that incorporates both self-attention and cross-attention mechanisms through a spatial-wise hybrid attention mechanism. This enhancement integrates spatial saliency cues into the global image embedding, improving accuracy and reliability. We also utilize the DINOv2 visual transformer for robust feature extraction. Extensive experiments on mainstream VPR benchmarks demonstrate that our method achieves superior performance while maintaining computational efficiency. Ablation studies and visualizations further validate the contributions of our attention mechanisms to the model’s performance improvement.

Real-time and efficient prediction of plausible trajectories of surrounding traffic agents is essential for autonomous driving. In reality, the motion of agents depends not only on their goal intents, but is also constrained by road topology. Especially for vehicles, lane geometry can significantly influence their future trajectories. In this paper, the constraining and guiding roles of lane networks are investigated, and a novel vehicle trajectory prediction model based on the goal lane segment is proposed. Specifically, a Dual-Stream Cross Attention Module (DSCAM) is developed that incorporates goal lane segment prediction into the process of collecting various interaction information. This achieves scene-consistent predictions while reducing resource consumption and inference latency. Then, learnable refinement tokens are used to adaptively refine coarse-grained trajectories during the feature decoding process, with the goal of improving prediction accuracy and ensuring temporal consistency. Extensive experiments with the Argoverse-1 and nuScenes datasets demonstrate that our model performs competitively and well-balanced. Notably, on the nuScenes, our model with only 0.44 million (M) parameters has an inference latency of less than six milliseconds (ms), making it ideal for large-scale production autonomous driving systems that require high computational resources, operational efficiency, and prediction performance.