Shuangjie Xu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Shuangjie Xu has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Maosheng Ye
HKUST
3 shared publications - Dit-Yan Yeung
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Sai Kung, Hong Kong
2 shared publications - Qifeng Chen
The Hong Kong University of Science and Technology, Hong Kong, Hong Kong
2 shared publications - Xiaoyi Zou
DEEPROUTE.AI
2 shared publications - Zhili Chen
HKUST
2 shared publications - Zian Qian
The Hong Kong University of Science and Technology, Hong Kong, Hong Kong
2 shared publications
Latest Publications
- 30 September 2024
- Published by Springer Nature
- p. 239-256
PPAD: Iterative Interactions of Prediction and Planning for End-to-End Autonomous Driving
- 30 September 2024
- Published by Springer Nature
- p. 239-256
PPAD: Iterative Interactions of Prediction and Planning for End-to-End Autonomous Driving
- 30 September 2024
- Published by Springer Nature
- p. 385-403
Learning High-Resolution Vector Representation from Multi-camera Images for 3D Object Detection
- 30 September 2024
- Published by Springer Nature
- p. 385-403
Learning High-Resolution Vector Representation from Multi-camera Images for 3D Object Detection
- Open Access
Learning High-resolution Vector Representation from Multi-Camera Images for 3D Object Detection
The Bird's-Eye-View (BEV) representation is a critical factor that directly impacts the 3D object detection performance, but the traditional BEV grid representation induces quadratic computational cost as the spatial resolution grows. To address this limitation, we present a new camera-based 3D object detector with high-resolution vector representation: VectorFormer. The presented high-resolution vector representation is combined with the lower-resolution BEV representation to efficiently exploit 3D geometry from multi-camera images at a high resolution through our two novel modules: vector scattering and gathering. To this end, the learned vector representation with richer scene contexts can serve as the decoding query for final predictions. We conduct extensive experiments on the nuScenes dataset and demonstrate state-of-the-art performance in NDS and inference time. Furthermore, we investigate query-BEV-based methods incorporated with our proposed vector representation and observe a consistent performance improvement.
Open AccessLearning High-resolution Vector Representation from Multi-Camera Images for 3D Object Detection
The Bird's-Eye-View (BEV) representation is a critical factor that directly impacts the 3D object detection performance, but the traditional BEV grid representation induces quadratic computational cost as the spatial resolution grows. To address this limitation, we present a new camera-based 3D object detector with high-resolution vector representation: VectorFormer. The presented high-resolution vector representation is combined with the lower-resolution BEV representation to efficiently exploit 3D geometry from multi-camera images at a high resolution through our two novel modules: vector scattering and gathering. To this end, the learned vector representation with richer scene contexts can serve as the decoding query for final predictions. We conduct extensive experiments on the nuScenes dataset and demonstrate state-of-the-art performance in NDS and inference time. Furthermore, we investigate query-BEV-based methods incorporated with our proposed vector representation and observe a consistent performance improvement.
- 2 September 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Engineering Management
- Vol. 70 (12) , 4120-4132
Computer Vision and Deep Learning to Manage Safety in Construction: Matching Images of Unsafe Behavior and Semantic Rules
The determination of people.s unsafe behavior from images in construction has been typically based on hand-made rule approaches, which renders it difficult to identify multiple acts of unsafe behavior within an image and accordingly apply safety rules. This article aims to develop a computer vision and deep learning method that can match images of people's unsafe behavior with semantic safety rules. Our proposed method consists of: 1) image feature representation; 2) safety rule feature representation; and 3) feature fusion similarity whereby unsafe behavior extracted from an image is matched with safety rules. We validate the effectiveness of our method using an image database of people's unsafe behavior from different sites associated with the construction of the Wuhan Metro Project (China). The results of our research explicitly demonstrate that our method is robust and can accurately recognize people's unsafe behavior and the corresponding safety rule that has been contravened. To this end, we suggest that construction organizations can use our method to manage safety better as part of a behavior-based safety strategy and thus prevent accidents.
- 2 September 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Engineering Management
- Vol. 70 (12) , 4120-4132
Computer Vision and Deep Learning to Manage Safety in Construction: Matching Images of Unsafe Behavior and Semantic Rules
The determination of people.s unsafe behavior from images in construction has been typically based on hand-made rule approaches, which renders it difficult to identify multiple acts of unsafe behavior within an image and accordingly apply safety rules. This article aims to develop a computer vision and deep learning method that can match images of people's unsafe behavior with semantic safety rules. Our proposed method consists of: 1) image feature representation; 2) safety rule feature representation; and 3) feature fusion similarity whereby unsafe behavior extracted from an image is matched with safety rules. We validate the effectiveness of our method using an image database of people's unsafe behavior from different sites associated with the construction of the Wuhan Metro Project (China). The results of our research explicitly demonstrate that our method is robust and can accurately recognize people's unsafe behavior and the corresponding safety rule that has been contravened. To this end, we suggest that construction organizations can use our method to manage safety better as part of a behavior-based safety strategy and thus prevent accidents.
- Open Access
- 18 May 2021
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 35 (3) , 2100-2108
Spatiotemporal Graph Neural Network based Mask Reconstruction for Video Object Segmentation
This paper addresses the task of segmenting class-agnostic objects in semi-supervised setting. Although previous detection based methods achieve relatively good performance, these approaches extract the best proposal by a greedy strategy, which may lose the local patch details outside the chosen candidate. In this paper, we propose a novel spatiotemporal graph neural network (STG-Net) to reconstruct more accurate masks for video object segmentation, which captures the local contexts by utilizing all proposals. In the spatial graph, we treat object proposals of a frame as nodes and represent their correlations with an edge weight strategy for mask context aggregation. To capture temporal information from previous frames, we use a memory network to refine the mask of current frame by retrieving historic masks in a temporal graph. The joint use of both local patch details and temporal relationships allow us to better address the challenges such as object occlusions and missing. Without online learning and fine-tuning, our STG-Net achieves state-of-the-art performance on four large benchmarks, demonstrating the effectiveness of the proposed approach.
- 18 May 2021
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 35 (3) , 2100-2108
Open AccessSpatiotemporal Graph Neural Network based Mask Reconstruction for Video Object Segmentation
This paper addresses the task of segmenting class-agnostic objects in semi-supervised setting. Although previous detection based methods achieve relatively good performance, these approaches extract the best proposal by a greedy strategy, which may lose the local patch details outside the chosen candidate. In this paper, we propose a novel spatiotemporal graph neural network (STG-Net) to reconstruct more accurate masks for video object segmentation, which captures the local contexts by utilizing all proposals. In the spatial graph, we treat object proposals of a frame as nodes and represent their correlations with an edge weight strategy for mask context aggregation. To capture temporal information from previous frames, we use a memory network to refine the mask of current frame by retrieving historic masks in a temporal graph. The joint use of both local patch details and temporal relationships allow us to better address the challenges such as object occlusions and missing. Without online learning and fine-tuning, our STG-Net achieves state-of-the-art performance on four large benchmarks, demonstrating the effectiveness of the proposed approach.
- Open Access
Crowd Counting via Hierarchical Scale Recalibration Network
The task of crowd counting is extremely challenging due to complicated difficulties, especially the huge variation in vision scale. Previous works tend to adopt a naive concatenation of multi-scale information to tackle it, while the scale shifts between the feature maps are ignored. In this paper, we propose a novel Hierarchical Scale Recalibration Network (HSRNet), which addresses the above issues by modeling rich contextual dependencies and recalibrating multiple scale-associated information. Specifically, a Scale Focus Module (SFM) first integrates global context into local features by modeling the semantic inter-dependencies along channel and spatial dimensions sequentially. In order to reallocate channel-wise feature responses, a Scale Recalibration Module (SRM) adopts a step-by-step fusion to generate final density maps. Furthermore, we propose a novel Scale Consistency loss to constrain that the scale-associated outputs are coherent with groundtruth of different scales. With the proposed modules, our approach can ignore various noises selectively and focus on appropriate crowd scales automatically. Extensive experiments on crowd counting datasets (ShanghaiTech, MALL, WorldEXPO'10, and UCSD) show that our HSRNet can deliver superior results over all state-of-the-art approaches. More remarkably, we extend experiments on an extra vehicle dataset, whose results indicate that the proposed model is generalized to other applications.
Open AccessCrowd Counting via Hierarchical Scale Recalibration Network
The task of crowd counting is extremely challenging due to complicated difficulties, especially the huge variation in vision scale. Previous works tend to adopt a naive concatenation of multi-scale information to tackle it, while the scale shifts between the feature maps are ignored. In this paper, we propose a novel Hierarchical Scale Recalibration Network (HSRNet), which addresses the above issues by modeling rich contextual dependencies and recalibrating multiple scale-associated information. Specifically, a Scale Focus Module (SFM) first integrates global context into local features by modeling the semantic inter-dependencies along channel and spatial dimensions sequentially. In order to reallocate channel-wise feature responses, a Scale Recalibration Module (SRM) adopts a step-by-step fusion to generate final density maps. Furthermore, we propose a novel Scale Consistency loss to constrain that the scale-associated outputs are coherent with groundtruth of different scales. With the proposed modules, our approach can ignore various noises selectively and focus on appropriate crowd scales automatically. Extensive experiments on crowd counting datasets (ShanghaiTech, MALL, WorldEXPO'10, and UCSD) show that our HSRNet can deliver superior results over all state-of-the-art approaches. More remarkably, we extend experiments on an extra vehicle dataset, whose results indicate that the proposed model is generalized to other applications.