LSLiangjing Shao

Liangjing Shao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Liangjing Shao has a total of 17 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • EndoLoc: Relative Pose Regression Framework With Transformation and Correlation Features for Visual Localization of Endoscope

    by Liangjing Shao, Benshuang Chen, Shuting Zhao, Fuming Yang, Xinrong Chen

    Real-time localization of endoscope is significant for the navigation and automation of endoscopic diagnosis and minimally invasive surgery. However, traditional localization based on optical tracking or magnetic tracking is easily influenced by occlusion or electromagnetic interference, while the implementation is complicated. Meanwhile, transformation and correlation information in image pairs are still ignored in existing visual localization methods for endoscopy. In this work, a novel relative pose regression framework is proposed for relative pose estimation and absolute pose tracking of endoscope based on endoscopic videos. Firstly, scene features and transformation features are respectively extracted from endoscopic observations and the corresponding optical flow by the proposed feature encoder based on gated convolution, which can prevent gradient vanishing when training the encoder from scratch on endoscopic data. Furthermore, a novel correlation module based on cross-attention is proposed to extract correlation features from two input images, which can capture more key features in endoscopic frames with more limited vision from local to global. Moreover, a novel pose decoder with upsampling and downsampling on the channel dimension is utilized to extract richer representation from the concatenated feature map for relative transformation vector prediction. The proposed method outperforms the state-of-the-art methods on the datasets from nasal endoscopy and colonoscopy, with an average localization error of less than 5%. The further experiments also demonstrate the efficiency of the proposed method. The demo videos of visual localization can be found on https://endoloc.netlify.app/


  • Open Access

    SSD-Poser: Avatar Pose Estimation with State Space Duality from Sparse Observations

    by Shuting Zhao, Linxin Bai, Liangjing Shao, Ye Zhang, Xinrong Chen

  • Open Access

    LL-MAROCO: A Large Language Model-Assisted Robotic System for Oral and Craniomaxillofacial Osteotomy

    by Lai Jiang, Liangjing Shao, Jinyang Wu, Xiaofeng Xu, Xinrong Chen, Shilei Zhang

    Oral and craniomaxillofacial bone deformities necessitate treatment through osteotomy. Robot-assisted osteotomy appears promising in oral and craniomaxillofacial surgery, but it lacks sufficient intelligence and comprehensive integration of navigation tracking with surgical planning. This study aims to develop an intelligent surgical robot, based on the large language model ChatGPT-4, to enable autonomous planning for oral and craniomaxillofacial osteotomies. An autonomous surgical planning system driven by ChatGPT-4 was developed. Surgical plans were autonomously generated based on expert-defined prompts and surgical objectives. A deep learning framework was employed to match navigation-generated visual data with textual planning outputs. The generated plans were subsequently converted into executable instructions for robotic surgery. System precision, execution accuracy, and usability were experimentally validated through common osteotomies. An anonymous Likert scale assessed operational efficiency. The proposed system achieved a trajectory planning accuracy of 0.24 mm and an average robotic execution accuracy of 1.46 mm. The completion rates for two representative procedures, Le Fort I osteotomy and genioplasty, were 87% and 92%, respectively. Survey results confirmed process feasibility. The integration of a large language model with surgical robot advances intelligent, precise, and safe oral and craniomaxillofacial osteotomy procedures.


  • EndoMODE: A Multimodal Visual Feature-Based Ego-Motion Estimation Framework for Monocular Odometry and Depth Estimation in Various Endoscopic Scenes

    by Liangjing Shao, Benshuang Chen, Shuting Zhao, Xinrong Chen

    Ego-motion estimation is a critical task for both monocular odometry and depth estimation, which are significant for navigation and scene perception in endoscopy. Most of existing ego-motion estimation frameworks only utilize a single modality of visual feature. In this work, a novel multimodal visual feature-based framework is proposed to perform real-time vision-based ego-motion estimation for visual odometry and depth estimation in endoscopic scenes. In the framework, to extract correlation information of two adjacent endoscopic frames, a channel attention-based module is proposed to integrate multidimensional features from the concatenated image and a feature interaction module is proposed to fuse scene features from two observations. Moreover, a novel pose decoder based on depthwise separable convolution is proposed to extract multiscale feature representation. Furthermore, a fully supervised pipeline and a self-supervised pipeline with the proposed framework are designed for monocular odometry and depth estimation, respectively. In the experiments, the proposed framework is compared with several advanced methods on five different datasets of various endoscopic scenes. Both quantitative and qualitative results demonstrate the pipeline with the proposed framework provides the most accurate monocular odometry and depth estimation in different kinds of endoscopic scenes.


  • Open Access

    SSD-Poser: Avatar Pose Estimation with State Space Duality from Sparse Observations

    by Shuting Zhao, Linxin Bai, Liangjing Shao, Ye Zhang, Xinrong Chen

    The growing applications of AR/VR increase the demand for real-time full-body pose estimation from Head-Mounted Displays (HMDs). Although HMDs provide joint signals from the head and hands, reconstructing a full-body pose remains challenging due to the unconstrained lower body. Recent advancements often rely on conventional neural networks and generative models to improve performance in this task, such as Transformers and diffusion models. However, these approaches struggle to strike a balance between achieving precise pose reconstruction and maintaining fast inference speed. To overcome these challenges, a lightweight and efficient model, SSD-Poser, is designed for robust full-body motion estimation from sparse observations. SSD-Poser incorporates a well-designed hybrid encoder, State Space Attention Encoders, to adapt the state space duality to complex motion poses and enable real-time realistic pose reconstruction. Moreover, a Frequency-Aware Decoder is introduced to mitigate jitter caused by variable-frequency motion signals, remarkably enhancing the motion smoothness. Comprehensive experiments on the AMASS dataset demonstrate that SSD-Poser achieves exceptional accuracy and computational efficiency, showing outstanding inference efficiency compared to state-of-the-art methods.

  • Estimation of Hand Pressure and Pose From RGB Images Based on Cross-Modal Cues

    by Wei Tang, Liangjing Shao, Xinrong Chen

    The hand is one of the most commonly used mediums for human-object interaction, and the sensing of hand force is an important research direction. Force measurement is traditionally realized by sensors, which is costly. Therefore, the visual-based method, which infers the hand pressure from RGB images, can be an effective way to estimate the contact force. To obtain the more accurate estimation of the hand tactile information through vision, the interaction of cross-modal information is designed into the model and the shape and gesture information needed is introduced for the hand pressure measurement. The experimental results show that the new model designed outperforms the state-of-the-art model by 0.825% to 5.93% in all important metrics on a large-scale dataset. In addition, the task of estimating hand posture and pressure from the same RGB image, as well as estimating the position of hand joints using regression models, can be obtained simultaneously.


  • 3-D Clothed Human Model Reconstruction Based on Single-View In-the-Wild Image Data

    by Liangjing Shao, Benshuang Chen, Ziqun Zhang, Xinrong Chen

    3D clothed human reconstruction has gained great development recently. However, many methods complete reconstruction based on model trained on the synthetic data with artificial appearance or the 3D scan data with high costs. The difference between synthetic data, 3D scan data and in-the-wild data severely restricts application of these methods in real world. Moreover, due to complex background and pose in the image, some researches on the 3D clothed human reconstruction from single-view in-the-wild images are facing challenges. In this paper, a modular model is designed to perform the 3D clothed human reconstruction with high quality based on one single-view in-the-wild image. In detail, the model consists of Cloth Encoder which extracts the feature of clothes, Body Encoder which obtains the parameters of human body and Cloth Generation Module which generates the clothes on the human model. Specifically, an adaptive integration of convolution neural network and self-attention is proposed to extract features of the input image. Moreover, to eliminate the distraction of the background, the segmentation of the input image is utilized to supervise our training process.The proposed method is compared with the mainstream methods on the two datasets, MSCOCO and 3DPW. The quantitative and qualitative results demonstrate that our method is more accurate in reconstructing clothed human model than the previous methods. Based on the experiment results, our work provides a more convenient and accurate 3D clothed human reconstruction technique in the real world.