Xiaobao Wei
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Xiaobao Wei has a total of 41 co-authors. Here are the top co-authors by number of shared publications:
- Ming Lu
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University, Beijing 100871, China
5 shared publications - Peng Chen
Chinese Academy of Sciences, Beijing, China
3 shared publications - Shanghang Zhang
Peking University, Beijing, China
3 shared publications - Anke Xue
Hangzhou Dianzi University, Hangzhou, China
2 shared publications - Hao Wang
Peking University, Beijing, China
2 shared publications Ming Lu
School of Computer Science, Peking University, Beijing, China
2 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 7196-7205
ThermalGaussian-X: RGB and Thermal Alignment via 3D Gaussian Splatting
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 7196-7205
Open AccessThermalGaussian-X: RGB and Thermal Alignment via 3D Gaussian Splatting
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 7186-7195
Dynamic Thermal Gaussians: Multimodal 4D Gaussian Splatting
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 7186-7195
Open AccessDynamic Thermal Gaussians: Multimodal 4D Gaussian Splatting
- Open Access
- 15 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1533-1541
SparseStreet: Sparse Gaussian Splatting for Real-Time Street Scene Simulation
While 3D Gaussian Splatting has shown promising results in street scene reconstruction, existing methods require massive numbers of Gaussian primitives to capture fine details, leading to prohibitive storage costs and slow rendering speeds. We observe that dynamic objects (e.g., vehicles and pedestrians) demand high-fidelity representations to maintain temporal consistency, while static background regions often contain substantial redundancy. Motivated by this, we propose SparseStreet, a general compression framework specifically designed for street scenes. First, we introduce a node-based learnable pruning strategy that systematically removes low-contributing Gaussian primitives while preserving visually critical regions. Second, after the scene representation stabilizes, we apply background compression, further reducing redundancy in static regions. Our method effectively preserves the geometry and appearance of dynamic objects while significantly reducing the total number of Gaussian primitives. Extensive experiments on the Waymo and nuScenes demonstrate that SparseStreet achieves up to 80% compression ratio with minimal quality degradation, enabling resource-efficient, high-fidelity dynamic scene reconstruction. Project website: https://sparsestreet.github.io/.
- 15 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 1533-1541
Open AccessSparseStreet: Sparse Gaussian Splatting for Real-Time Street Scene Simulation
While 3D Gaussian Splatting has shown promising results in street scene reconstruction, existing methods require massive numbers of Gaussian primitives to capture fine details, leading to prohibitive storage costs and slow rendering speeds. We observe that dynamic objects (e.g., vehicles and pedestrians) demand high-fidelity representations to maintain temporal consistency, while static background regions often contain substantial redundancy. Motivated by this, we propose SparseStreet, a general compression framework specifically designed for street scenes. First, we introduce a node-based learnable pruning strategy that systematically removes low-contributing Gaussian primitives while preserving visually critical regions. Second, after the scene representation stabilizes, we apply background compression, further reducing redundancy in static regions. Our method effectively preserves the geometry and appearance of dynamic objects while significantly reducing the total number of Gaussian primitives. Extensive experiments on the Waymo and nuScenes demonstrate that SparseStreet achieves up to 80% compression ratio with minimal quality degradation, enabling resource-efficient, high-fidelity dynamic scene reconstruction. Project website: https://sparsestreet.github.io/.
- 1 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 32 (5) , 4228-4238
RSATalker: Realistic Socially-Aware Talking Head Generation for Multi-Turn Conversation
Talking head generation is increasingly important in virtual reality (VR), especially for social scenarios involving multi-turn conversation. Existing approaches face notable limitations: mesh-based 3D methods can model dual-person dialogue but lack realistic textures, large-model-based 2D methods produce natural appearances but incur prohibitive computational costs. Recently, 3D Gaussian Splatting (3DGS)-based methods achieve efficient and realistic rendering but remain speaker-only and ignore social relationships. We introduce RSATalker, the first framework that leverages 3DGS for realistic and socially-aware talking head generation, with support for multi-turn conversation. Our method first drives mesh-based 3D facial motion from speech, then binds 3D Gaussians to mesh facets to render high-fidelity 2D avatar videos. To capture interpersonal dynamics, we propose a socially-aware module that encodes social relationships, including blood and non-blood as well as equal and unequal, into high-level embeddings through a learnable query mechanism. We design a three-stage training paradigm and construct the RSATalker dataset with speech-mesh-image triplets annotated with social relationships. Our method supports applications such as VR telepresence, social VR, and embodied conversational agents. The socially-aware conditioning can also be extended to other human motion generation tasks. Extensive experiments demonstrate that RSATalker achieves state-of-the-art performance in both realism and social awareness. The code and dataset will be released.
- 1 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 32 (5) , 4228-4238
RSATalker: Realistic Socially-Aware Talking Head Generation for Multi-Turn Conversation
Talking head generation is increasingly important in virtual reality (VR), especially for social scenarios involving multi-turn conversation. Existing approaches face notable limitations: mesh-based 3D methods can model dual-person dialogue but lack realistic textures, large-model-based 2D methods produce natural appearances but incur prohibitive computational costs. Recently, 3D Gaussian Splatting (3DGS)-based methods achieve efficient and realistic rendering but remain speaker-only and ignore social relationships. We introduce RSATalker, the first framework that leverages 3DGS for realistic and socially-aware talking head generation, with support for multi-turn conversation. Our method first drives mesh-based 3D facial motion from speech, then binds 3D Gaussians to mesh facets to render high-fidelity 2D avatar videos. To capture interpersonal dynamics, we propose a socially-aware module that encodes social relationships, including blood and non-blood as well as equal and unequal, into high-level embeddings through a learnable query mechanism. We design a three-stage training paradigm and construct the RSATalker dataset with speech-mesh-image triplets annotated with social relationships. Our method supports applications such as VR telepresence, social VR, and embodied conversational agents. The socially-aware conditioning can also be extended to other human motion generation tasks. Extensive experiments demonstrate that RSATalker achieves state-of-the-art performance in both realism and social awareness. The code and dataset will be released.
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 945-954
MixedGaussianAvatar: Realistically and Geometrically Accurate Head Avatar via Mixed 2D-3D Gaussians
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 945-954
Open AccessMixedGaussianAvatar: Realistically and Geometrically Accurate Head Avatar via Mixed 2D-3D Gaussians
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 925-934
EmbodiedOcc++: Boosting Embodied 3D Occupancy Prediction with Plane Regularization and Uncertainty Sampler
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 925-934
Open AccessEmbodiedOcc++: Boosting Embodied 3D Occupancy Prediction with Plane Regularization and Uncertainty Sampler
- 2 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 34 (10577149) , 3377-3388
PLGS: Robust Panoptic Lifting With 3D Gaussian Splatting
Previous methods utilize the Neural Radiance Field (NeRF) for panoptic lifting, while their training and rendering speed are unsatisfactory. In contrast, 3D Gaussian Splatting (3DGS) has emerged as a prominent technique due to its rapid training and rendering speed. However, unlike NeRF, the conventional 3DGS may not satisfy the basic smoothness assumption as it does not rely on any parameterized structures to render (e.g., MLPs). Consequently, the conventional 3DGS is, in nature, more susceptible to noisy 2D mask supervision. In this paper, we propose a new method called PLGS that enables 3DGS to generate consistent panoptic segmentation masks from noisy 2D segmentation masks while maintaining superior efficiency compared to NeRF-based methods. Specifically, we build a panoptic-aware structured 3D Gaussian model to introduce smoothness and design effective noise reduction strategies. For the semantic field, instead of initialization with structure from motion, we construct reliable semantic anchor points to initialize the 3D Gaussians. We then use these anchor points as smooth regularization during training. Additionally, we present a self-training approach using pseudo labels generated by merging the rendered masks with the noisy masks to enhance the robustness of PLGS. For the instance field, we project the 2D instance masks into 3D space and match them with oriented bounding boxes to generate cross-view consistent instance masks for supervision. Experiments on various benchmarks demonstrate that our method outperforms previous state-of-the-art methods in terms of both segmentation quality and speed.
- 2 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 34 (10577149) , 3377-3388
PLGS: Robust Panoptic Lifting With 3D Gaussian Splatting
Previous methods utilize the Neural Radiance Field (NeRF) for panoptic lifting, while their training and rendering speed are unsatisfactory. In contrast, 3D Gaussian Splatting (3DGS) has emerged as a prominent technique due to its rapid training and rendering speed. However, unlike NeRF, the conventional 3DGS may not satisfy the basic smoothness assumption as it does not rely on any parameterized structures to render (e.g., MLPs). Consequently, the conventional 3DGS is, in nature, more susceptible to noisy 2D mask supervision. In this paper, we propose a new method called PLGS that enables 3DGS to generate consistent panoptic segmentation masks from noisy 2D segmentation masks while maintaining superior efficiency compared to NeRF-based methods. Specifically, we build a panoptic-aware structured 3D Gaussian model to introduce smoothness and design effective noise reduction strategies. For the semantic field, instead of initialization with structure from motion, we construct reliable semantic anchor points to initialize the 3D Gaussians. We then use these anchor points as smooth regularization during training. Additionally, we present a self-training approach using pseudo labels generated by merging the rendered masks with the noisy masks to enhance the robustness of PLGS. For the instance field, we project the 2D instance masks into 3D space and match them with oriented bounding boxes to generate cross-view consistent instance masks for supervision. Experiments on various benchmarks demonstrate that our method outperforms previous state-of-the-art methods in terms of both segmentation quality and speed.
- 3 November 2024
- Published by Springer Nature
- p. 90-107
I-MedSAM: Implicit Medical Image Segmentation with Segment Anything
- 3 November 2024
- Published by Springer Nature
- p. 90-107
I-MedSAM: Implicit Medical Image Segmentation with Segment Anything
- 26 July 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 194-199
Time-varying group formation-tracking control for heterogeneous multi-agent systems with switching topologies and time-varying delays
Group formation-tracking problem for heterogeneous multi-agent systems (HMASs) with both switching networks and communication delays is investigated in this paper. In order to achieve different tasks, the agents are classified into various groups. The followers are allowed to realize the formation and track the trajectory of the leader in each group. Firstly, by utilizing the consensus control, an observer is proposed to estimate the state of the leader. Then, an observer-based control protocol is put forward to solve the group formation-tracking problem with both communication delays and switching networks. Moreover, an algorithm to determine the gain feedbacks is demonstrated, in the meantime, the observer’s error systems as well as the group formation-tracking error systems are proved to be convergent. Finally, an example in the simulation prat is presented to verify the theoretical results.
- 26 July 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 194-199
Time-varying group formation-tracking control for heterogeneous multi-agent systems with switching topologies and time-varying delays
Group formation-tracking problem for heterogeneous multi-agent systems (HMASs) with both switching networks and communication delays is investigated in this paper. In order to achieve different tasks, the agents are classified into various groups. The followers are allowed to realize the formation and track the trajectory of the leader in each group. Firstly, by utilizing the consensus control, an observer is proposed to estimate the state of the leader. Then, an observer-based control protocol is put forward to solve the group formation-tracking problem with both communication delays and switching networks. Moreover, an algorithm to determine the gain feedbacks is demonstrated, in the meantime, the observer’s error systems as well as the group formation-tracking error systems are proved to be convergent. Finally, an example in the simulation prat is presented to verify the theoretical results.