SSShuwei Shao

Shuwei Shao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Shuwei Shao has a total of 47 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Beyond Foundation Models: Distilling Geometric Priors for Lightweight Monocular Depth Estimation in Endoscopy

    by Kejin Zhu, Shuwei Shao, Yongming Yang, Zhongyu Tian, Baochang Zhang, Zhe Min

    In recent times, geometric foundation models have demonstrated remarkable performance in depth estimation tasks, benefiting from exposure to large-scale data that enables the learning of intricate geometric structures and spatial dependencies. However, their large parameter sizes and high computational complexity pose significant challenges in meeting the efficiency requirements of downstream surgical applications. Consequently, the design of a high-performance yet lightweight monocular depth estimator has become a focal point of research. To this end, we harness the rich geometric priors encoded in geometric foundation models and introduce a novel trinity distillation scheme that transfers geometric knowledge across three complementary dimensions, namely spatial, spectral and gradient, into a compact depth estimator. To further enhance prediction quality, we develop a semantic distribution alignment strategy to effectively suppress pseudo-texture artifacts arising from the limited semantic representation capability of the lightweight estimator. Extensive experiments on the SCARED, SERV-CT, Hamlyn, and C3VD datasets demonstrate that the proposed method either surpasses or achieves comparable performance to previous state-of-the-art competitors, with a smaller model size and reduced computational overhead. Code will be available at: https://github.com/ShuweiShao/LiteNet.

  • Endo-DFD: Deep Flow-to-Depth in Endoscopy With Uncertainty-Driven Recurrent Refinement

    by Yongming Yang, Peng Wang, Shuwei Shao, Ziliang Song, Hesheng Wang, Hao Liu

    Dense depth estimation from monocular video frames provides accurate intraoperative 3D point clouds for augmented reality surgical navigation. However, multi-frame depth estimation within the endoscopic scene suffers from the inaccurate matching of geometric relationships under subtle camera movements. To address this challenge, we propose a novel deep flow-to-depth method with uncertainty-driven recurrent refinement. Specifically, a pixel-wise uncertainty-driven optimizer is proposed to iteratively refine depth predictions instead of the traditional cost volume, effectively reducing depth errors caused by low textures and illumination variations. Furthermore, a two-view geometry with motion simulator is employed to augment the optical flow network’s ability to explore temporal correlations under subtle camera movements. Extensive cross-datasets experiments demonstrate the superior accuracy and generalization of our method, achieving RMSE values of 6.566 on C3VD, 7.972 on EndoSLAM, and 16.697 on ColonDepth datasets. Qualitative evaluation conducted on clinical dataset demonstrates the promising ability of our method to accurately estimate realistic anatomical structures in real-world scenarios. The code will be accessible at the following link: https://github. com/ILSR-SIA/Endo-DFD.

  • GMDepth: Depth Completion via Global Dense-to-Sparse Matching

    by Shuwei Shao, Zhongcai Pei, Weihai Chen, Zhong Liu, Zhengguo Li

    Depth completion plays a key role in sensor-based measurement systems, as it facilitates the recovery of accurate and complete depth measurements from sparsely sampled sensor data. In this paper, we revamp the prevailing depth completion paradigms such as classification-regression by formulating the task as a global dense-to-sparse matching problem. The proposed formulation completes missing pixels by matching each one to the optimal depth candidates among all sparse points. In order to instantiate it, we introduce a GMDepth framework, which involves three main components: an encoder-decoder network for feature extraction, successive cross-attention layers for feature enhancement, and a correlation-softmax layer for the global dense-to-sparse matching. Furthermore, we propose a self-induced calibration map to strengthen the robustness and stability of matching process against sensor noise. Extensive experiments indicate that our method delivers superior performance compared to existing state-of-the-art methods, offering a promising solution for depth measurement tasks.

  • Open Access

    HRGS: Hierarchical Gaussian Splatting for Memory-Efficient High-Resolution 3D Reconstruction

    by Changbai Li, Haodong Zhu, Hanlin Chen, Juan Zhang, Tongfei Chen, Shuo Yang, Shuwei Shao, Wenhao Dong, Baochang Zhang

    3D Gaussian Splatting (3DGS) has made significant strides in real-time 3D scene reconstruction, but faces memory scalability issues in high-resolution scenarios. To address this, we propose Hierarchical Gaussian Splatting (HRGS), a memory-efficient framework with hierarchical block-level optimization. First, we generate a global, coarse Gaussian representation from low-resolution data. Then, we partition the scene into multiple blocks, refining each block with high-resolution data. The partitioning involves two steps: Gaussian partitioning, where irregular scenes are normalized into a bounded cubic space with a uniform grid for task distribution, and training data partitioning, where only relevant observations are retained for each block. By guiding block refinement with the coarse Gaussian prior, we ensure seamless Gaussian fusion across adjacent blocks. To reduce computational demands, we introduce Importance-Driven Gaussian Pruning (IDGP), which computes importance scores for each Gaussian and removes those with minimal contribution, speeding up convergence and reducing memory usage. Additionally, we incorporate normal priors from a pretrained model to enhance surface reconstruction quality. Our method enables high-quality, high-resolution 3D scene reconstruction even under memory constraints. Extensive experiments on three benchmarks show that HRGS achieves state-of-the-art performance in high-resolution novel view synthesis (NVS) and surface reconstruction tasks.

  • MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Model

    by Shuwei Shao, Zhongcai Pei, Weihai Chen, Dingchi Sun, Peter C. Y. Chen, Zhengguo Li

    Over the past few years, self-supervised monocular depth estimation has received widespread attention. Most efforts focus on designing different types of network architectures and loss functions or handling edge cases, for example, occlusion and dynamic objects. In this work, we take another path and propose a novel conditional diffusion-based generative framework for self-supervised monocular depth estimation, dubbed MonoDiffusion. Because the depth ground-truth is unavailable in a self-supervised setting, we develop a new pseudo ground-truth diffusion process to assist the diffusion for training. Instead of diffusing at a fixed high resolution, we perform diffusion in a coarse-to-fine manner that allows for faster inference time without sacrificing accuracy or even better accuracy. Furthermore, we develop a simple yet effective contrastive depth reconstruction mechanism to enhance the denoising ability of model. It is worth noting that the proposed MonoDiffusion has the property of naturally acquiring the depth uncertainty that is essential to be implemented in safety-critical cases. Extensive experiments on the KITTI, Make3D and DIML datasets indicate that our MonoDiffusion outperforms prior state-of-the-art self-supervised competitors. The source code will be publicly available upon the acceptance.


  • Open Access

    Digging into Contrastive Learning for Robust Depth Estimation with Diffusion Models

    by Jiyuan Wang, Chunyu Lin, Lang Nie, Kang Liao, Shuwei Shao, Yao Zhao

    Recently, diffusion-based depth estimation methods have drawn widespread attention due to their elegant denoising patterns and promising performance. However, they are typically unreliable under adverse conditions prevalent in real-world scenarios, such as rainy, snowy, etc. In this paper, we propose a novel robust depth estimation method called D4RD, featuring a custom contrastive learning mode tailored for diffusion models to mitigate performance degradation in complex environments. Concretely, we integrate the strength of knowledge distillation into contrastive learning, building the 'trinity' contrastive scheme. This scheme utilizes the sampled noise of the forward diffusion process as a natural reference, guiding the predicted noise in diverse scenes toward a more stable and precise optimum. Moreover, we extend noise-level trinity to encompass more generic feature and image levels, establishing a multi-level contrast to distribute the burden of robust perception across the overall network. Before addressing complex scenarios, we enhance the stability of the baseline diffusion model with three straightforward yet effective improvements, which facilitate convergence and remove depth outliers. Extensive experiments demonstrate that D4RD surpasses existing state-of-the-art solutions on synthetic corruption datasets and real-world weather conditions. Source code and data are available at https://github.com/wangjiyuan9/D4RD.


  • Open Access

    SPDepth: Enhancing Self-Supervised Indoor Monocular Depth Estimation via Self-Propagation

    by Xiaotong Guo, Huijie Zhao, Shuwei Shao, Xudong Li, Baochang Zhang, Na Li

    Due to the existence of low-textured areas in indoor scenes, some self-supervised depth estimation methods have specifically designed sparse photometric consistency losses and geometry-based losses. However, some of the loss terms cannot supervise all the pixels, which limits the performance of these methods. Some approaches introduce an additional optical flow network to provide dense correspondences supervision, but overload the loss function. In this paper, we propose to perform depth self-propagation based on feature self-similarities, where high-accuracy depths are propagated from supervised pixels to unsupervised ones. The enhanced self-supervised indoor monocular depth estimation network is called SPDepth. Since depth self-similarities are significant in a local range, a local window self-attention module is embedded at the end of the network to propagate depths in a window. The depth of a pixel is weighted using the feature correlation scores with other pixels in the same window. The effectiveness of self-propagation mechanism is demonstrated in the experiments on the NYU Depth V2 dataset. The root-mean-squared error of SPDepth is 0.585 and the δ1 accuracy is 77.6%. Zero-shot generalization studies are also conducted on the 7-Scenes dataset and provide a more comprehensive analysis about the application characteristics of SPDepth.


  • Open Access

    SIM-MultiDepth: Self-Supervised Indoor Monocular Multi-Frame Depth Estimation Based on Texture-Aware Masking

    by Xiaotong Guo, Huijie Zhao, Shuwei Shao, Xudong Li, Baochang Zhang, Na Li

    Self-supervised monocular depth estimation methods have become the focus of research since ground truth data are not required. Current single-image-based works only leverage appearance-based features, thus achieving a limited performance. Deep learning based multiview stereo works facilitate the research on multi-frame depth estimation methods. Some multi-frame methods build cost volumes and take multiple frames as inputs at the time of test to fully utilize geometric cues between adjacent frames. Nevertheless, low-textured regions, which are dominant in indoor scenes, tend to cause unreliable depth hypotheses in the cost volume. Few self-supervised multi-frame methods have been used to conduct research on the issue of low-texture areas in indoor scenes. To handle this issue, we propose SIM-MultiDepth, a self-supervised indoor monocular multi-frame depth estimation framework. A self-supervised single-frame depth estimation network is introduced to learn the relative poses and supervise the multi-frame depth learning. A texture-aware depth consistency loss is designed considering the calculation of the patch-based photometric loss. Only the areas where multi-frame depth prediction is considered unreliable in low-texture regions are supervised by the single-frame network. This approach helps improve the depth estimation accuracy. The experimental results on the NYU Depth V2 dataset validate the effectiveness of SIM-MultiDepth. The zero-shot generalization studies on the 7-Scenes and Campus Indoor datasets aid in the analysis of the application characteristics of SIM-MultiDepth.


  • NDDepth: Normal-Distance Assisted Monocular Depth Estimation and Completion

    by Shuwei Shao, Zhongcai Pei, Weihai Chen, Peter C. Y. Chen, Zhengguo Li

    Over the past few years, monocular depth estimation and completion have been paid more and more attention from the computer vision community because of their widespread applications. In this paper, we introduce novel physics (geometry)-driven deep learning frameworks for these two tasks by assuming that 3D scenes are constituted with piece-wise planes. Instead of directly estimating the depth map or completing the sparse depth map, we propose to estimate the surface normal and plane-to-origin distance maps or complete the sparse surface normal and distance maps as intermediate outputs. To this end, we develop a normal-distance head that outputs pixel-level surface normal and distance. Afterthat, the surface normal and distance maps are regularized by a developed plane-aware consistency constraint, which are then transformed into depth maps. Furthermore, we integrate an additional depth head to strengthen the robustness of the proposed frameworks. Extensive experiments on the NYU-Depth-v2, KITTI and SUN RGB-D datasets demonstrate that our method exceeds in performance prior state-of-the-art monocular depth estimation and completion competitors.


  • Open Access

    $\mathrm{F^2Depth}$: Self-supervised Indoor Monocular Depth Estimation via Optical Flow Consistency and Feature Map Synthesis

    by Xiaotong Guo, Huijie Zhao, Shuwei Shao, Xudong Li, Baochang Zhang

    Self-supervised monocular depth estimation methods have been increasingly given much attention due to the benefit of not requiring large, labelled datasets. Such self-supervised methods require high-quality salient features and consequently suffer from severe performance drop for indoor scenes, where low-textured regions dominant in the scenes are almost indiscriminative. To address the issue, we propose a self-supervised indoor monocular depth estimation framework called $\mathrm{F^2Depth}$. A self-supervised optical flow estimation network is introduced to supervise depth learning. To improve optical flow estimation performance in low-textured areas, only some patches of points with more discriminative features are adopted for finetuning based on our well-designed patch-based photometric loss. The finetuned optical flow estimation network generates high-accuracy optical flow as a supervisory signal for depth estimation. Correspondingly, an optical flow consistency loss is designed. Multi-scale feature maps produced by finetuned optical flow estimation network perform warping to compute feature map synthesis loss as another supervisory signal for depth learning. Experimental results on the NYU Depth V2 dataset demonstrate the effectiveness of the framework and our proposed losses. To evaluate the generalization ability of our $\mathrm{F^2Depth}$, we collect a Campus Indoor depth dataset composed of approximately 1500 points selected from 99 images in 18 scenes. Zero-shot generalization experiments on 7-Scenes dataset and Campus Indoor achieve $δ_1$ accuracy of 75.8% and 76.0% respectively. The accuracy results show that our model can generalize well to monocular images captured in unknown indoor scenes.