Ziyi Shen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Ziyi Shen has a total of 56 co-authors. Here are the top co-authors by number of shared publications:
- Dean C. Barratt
University College London
4 shared publications - Yipeng Hu
UCL Hawkes Institute, University College London, London, UK
4 shared publications - Tingfa Xu
Beijing Institute of Technology
3 shared publications - Jianbing Shen
University of Macau
2 shared publications Jie Guo
School of Optics and Photonics Beijing Institute of Technology China
2 shared publications- Ling Shao
University of Chinese Academy of Sciences
2 shared publications
Latest Publications
- 17 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35 (10577149) , 4383-4396
PromptReg: Interactive Registration by “Corresponding Prompts” for Segment Anything Model (SAM)
Effectively establishing correspondence between two images is at the centre of image registration methods. Spatially omnipresent representations, including dense displacement fields (DDFs) and spatial (non-)rigid transformations, have been used to parameterise such correspondence. Alternatively, region-based representation uses paired regions of interest (ROIs) to represent region-level correspondence, while retaining its local and dense representation capability at pixel/voxel level if required. Thus, registration can be re-envisioned as a problem of segmenting corresponding paired ROIs in the to-be-registered images. In this work, we utilize models such as SAM, which are pre-trained on substantive datasets, to segment ROIs of the same class from two images, for a new training-free, non-iterative registration algorithm. First, a “corresponding prompt problem” is posed to find a corresponding Prompt Y on Image Y, given any vision Prompt X on Image X, such that the two respectively prompt-conditioned segmentations are a pair of corresponding ROIs from the two images. Second, we propose an “inverse prompt” solution to the corresponding prompt problem, by inverting Prompt X to the Image Y prompt space, where the Jacobian of prototypical features is used. Third, we propose a new registration algorithm that identifies multiple paired corresponding ROIs, by marginalizing the inverted Prompt X over both prompt and spatial spaces, random sampling Prompt X and spatial warping Image X. Comprehensive experiments were conducted on five applications of registering 3D prostate MR, 3D abdomen CT, 3D lung CT, 2D histopathology and, as a non-medical example, 2D aerial images. Based on metrics including Dice and target registration errors on anatomical structures, the proposed registration outperforms both intensity-based iterative algorithms and learning-based networks, even yielding competitive performance with weakly-supervised registration which requires fully-segmented training data.
- 17 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35 (10577149) , 4383-4396
PromptReg: Interactive Registration by “Corresponding Prompts” for Segment Anything Model (SAM)
Effectively establishing correspondence between two images is at the centre of image registration methods. Spatially omnipresent representations, including dense displacement fields (DDFs) and spatial (non-)rigid transformations, have been used to parameterise such correspondence. Alternatively, region-based representation uses paired regions of interest (ROIs) to represent region-level correspondence, while retaining its local and dense representation capability at pixel/voxel level if required. Thus, registration can be re-envisioned as a problem of segmenting corresponding paired ROIs in the to-be-registered images. In this work, we utilize models such as SAM, which are pre-trained on substantive datasets, to segment ROIs of the same class from two images, for a new training-free, non-iterative registration algorithm. First, a “corresponding prompt problem” is posed to find a corresponding Prompt Y on Image Y, given any vision Prompt X on Image X, such that the two respectively prompt-conditioned segmentations are a pair of corresponding ROIs from the two images. Second, we propose an “inverse prompt” solution to the corresponding prompt problem, by inverting Prompt X to the Image Y prompt space, where the Jacobian of prototypical features is used. Third, we propose a new registration algorithm that identifies multiple paired corresponding ROIs, by marginalizing the inverted Prompt X over both prompt and spatial spaces, random sampling Prompt X and spatial warping Image X. Comprehensive experiments were conducted on five applications of registering 3D prostate MR, 3D abdomen CT, 3D lung CT, 2D histopathology and, as a non-medical example, 2D aerial images. Based on metrics including Dice and target registration errors on anatomical structures, the proposed registration outperforms both intensity-based iterative algorithms and learning-based networks, even yielding competitive performance with weakly-supervised registration which requires fully-segmented training data.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. PP, 1
SAMReg: SAM-enabled Image Registration with ROI-based Correspondence
This paper describes a new spatial correspondence representation based on paired regions-of-interest (ROIs), for medical image registration. The distinct properties of the proposed ROI-based correspondence are discussed, in the context of potential benefits in clinical applications following image registration, compared with alternative correspondence-representing approaches, such as those based on sampled displacements and spatial transformation functions. These benefits include a clear connection between learning-based image registration and segmentation, which in turn motivates two cases of image registration approaches using (pre-)trained segmentation networks. Based on the segment anything model (SAM), a vision foundation model for segmentation, we develop a new registration algorithm SAMReg, which does not require any training (or training data), gradient-based fine-tuning or prompt engineering. The proposed SAMReg models are evaluated across five real-world applications, including intra-subject registration tasks with cardiac MR and lung CT, challenging inter-subject registration scenarios with prostate MR and retinal imaging, and an additional evaluation with a non-clinical example with aerial image registration. The proposed methods outperform both intensity-based iterative algorithms and DDF-predicting learning-based networks across tested metrics including Dice and target registration errors on anatomical structures, and further demonstrates competitive performance compared to weakly-supervised registration approaches that rely on fully-segmented training data. Open source code and examples are available at SAMReg.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. PP, 1
SAMReg: SAM-enabled Image Registration with ROI-based Correspondence
This paper describes a new spatial correspondence representation based on paired regions-of-interest (ROIs), for medical image registration. The distinct properties of the proposed ROI-based correspondence are discussed, in the context of potential benefits in clinical applications following image registration, compared with alternative correspondence-representing approaches, such as those based on sampled displacements and spatial transformation functions. These benefits include a clear connection between learning-based image registration and segmentation, which in turn motivates two cases of image registration approaches using (pre-)trained segmentation networks. Based on the segment anything model (SAM), a vision foundation model for segmentation, we develop a new registration algorithm SAMReg, which does not require any training (or training data), gradient-based fine-tuning or prompt engineering. The proposed SAMReg models are evaluated across five real-world applications, including intra-subject registration tasks with cardiac MR and lung CT, challenging inter-subject registration scenarios with prostate MR and retinal imaging, and an additional evaluation with a non-clinical example with aerial image registration. The proposed methods outperform both intensity-based iterative algorithms and DDF-predicting learning-based networks across tested metrics including Dice and target registration errors on anatomical structures, and further demonstrates competitive performance compared to weakly-supervised registration approaches that rely on fully-segmented training data. Open source code and examples are available at SAMReg.
- 24 February 2024
- Published by Elsevier in Journal of Energy Storage
- Vol. 85, 110976
Ultrasound overcomes dendrite puncture in Li metal batteries
- 24 February 2024
- Published by Elsevier in Journal of Energy Storage
- Vol. 85, 110976
Ultrasound overcomes dendrite puncture in Li metal batteries
- 19 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biomedical Engineering
- Vol. 71 (3) , 1033-1042
Long-Term Dependency for 3D Reconstruction of Freehand Ultrasound Without External Tracker
Objective: Reconstructing freehand ultrasound in 3D without any external tracker has been a long-standing challenge in ultrasound-assisted procedures. We aim to define new ways of parameterising long-term dependencies, and evaluate the performance. Methods: First, long-term dependency is encoded by transformation positions within a frame sequence. This is achieved by combining a sequence model with a multi-transformation prediction. Second, two dependency factors are proposed, anatomical image content and scanning protocol, for contributing towards accurate reconstruction. Each factor is quantified experimentally by reducing respective training variances. Results: 1) The added long-term dependency up to 400 frames at 20 frames per second (fps) indeed improved reconstruction, with an up to 82.4% lowered accumulated error, compared with the baseline performance. The improvement was found to be dependent on sequence length, transformation interval and scanning protocol and, unexpectedly, not on the use of recurrent networks with long-short term modules; 2) Decreasing either anatomical or protocol variance in training led to poorer reconstruction accuracy. Interestingly, greater performance was gained from representative protocol patterns, than from representative anatomical features. Conclusion: The proposed algorithm uses hyperparameter tuning to effectively utilise long-term dependency. The proposed dependency factors are of practical significance in collecting diverse training data, regulating scanning protocols and developing efficient networks. Significance: The proposed new methodology with publicly available volunteer data and code for parametersing the long-term dependency, experimentally shown to be valid sources of performance improvement, which could potentially lead to better model development and practical optimisation of the reconstruction application.
- 19 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Biomedical Engineering
- Vol. 71 (3) , 1033-1042
Long-Term Dependency for 3D Reconstruction of Freehand Ultrasound Without External Tracker
Objective: Reconstructing freehand ultrasound in 3D without any external tracker has been a long-standing challenge in ultrasound-assisted procedures. We aim to define new ways of parameterising long-term dependencies, and evaluate the performance. Methods: First, long-term dependency is encoded by transformation positions within a frame sequence. This is achieved by combining a sequence model with a multi-transformation prediction. Second, two dependency factors are proposed, anatomical image content and scanning protocol, for contributing towards accurate reconstruction. Each factor is quantified experimentally by reducing respective training variances. Results: 1) The added long-term dependency up to 400 frames at 20 frames per second (fps) indeed improved reconstruction, with an up to 82.4% lowered accumulated error, compared with the baseline performance. The improvement was found to be dependent on sequence length, transformation interval and scanning protocol and, unexpectedly, not on the use of recurrent networks with long-short term modules; 2) Decreasing either anatomical or protocol variance in training led to poorer reconstruction accuracy. Interestingly, greater performance was gained from representative protocol patterns, than from representative anatomical features. Conclusion: The proposed algorithm uses hyperparameter tuning to effectively utilise long-term dependency. The proposed dependency factors are of practical significance in collecting diverse training data, regulating scanning protocols and developing efficient networks. Significance: The proposed new methodology with publicly available volunteer data and code for parametersing the long-term dependency, experimentally shown to be valid sources of performance improvement, which could potentially lead to better model development and practical optimisation of the reconstruction application.
- 17 September 2022
- Published by Springer Nature
- p. 237-247
Collaborative Quantization Embeddings for Intra-subject Prostate MR Image Registration
- 17 September 2022
- Published by Springer Nature
- p. 237-247
Collaborative Quantization Embeddings for Intra-subject Prostate MR Image Registration
- 10 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-5
Equivalence of nanoindentations by the load and displacement controlled manners
Nanoindentation experiments have been attracting more attentions to in situ test the mechanical properties of die-attach materials due to the localized loading features. Load controlled manner is usually performed to maintain the indentation strain rate, which results in great challenges for finite element (FE) simulations with the indenter controlled in a displacement manner. In this paper, a validated dimensionless method is adopted to describe the applied load–penetration depth (P–h) curve, where the loading curvature represents the mechanical properties of the materials of interest. Based on a typical FE simulation by using the Berkovich indenter, the relationship between load and displacement controlled methods is investigated. The reaction force on the indenter can be measured from the FE simulations controlled by the applied displacement of the indenter. Meanwhile, the P–h curve of indentations controlled by load can be predicted by the proposed method with the provided loading curvature. As a calibration of the time increment Δt, the calculation results show that the smaller the Δt value is, the influence on the predictions can be ignored. In other words, the predictions of penetration depth can be independent of the Δt value and thus reflect the objective essence of loading process. By taking Δt value as 0.001s for three materials with different constitutive properties, the P–h curves under load and displacement control manners are compared, and excellent agreement is found. The proposed method provides a new and elegant way to realize equivalent load control by displacement control of the indenter for nanoindentation simulations.
- 10 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-5
Equivalence of nanoindentations by the load and displacement controlled manners
Nanoindentation experiments have been attracting more attentions to in situ test the mechanical properties of die-attach materials due to the localized loading features. Load controlled manner is usually performed to maintain the indentation strain rate, which results in great challenges for finite element (FE) simulations with the indenter controlled in a displacement manner. In this paper, a validated dimensionless method is adopted to describe the applied load–penetration depth (P–h) curve, where the loading curvature represents the mechanical properties of the materials of interest. Based on a typical FE simulation by using the Berkovich indenter, the relationship between load and displacement controlled methods is investigated. The reaction force on the indenter can be measured from the FE simulations controlled by the applied displacement of the indenter. Meanwhile, the P–h curve of indentations controlled by load can be predicted by the proposed method with the provided loading curvature. As a calibration of the time increment Δt, the calculation results show that the smaller the Δt value is, the influence on the predictions can be ignored. In other words, the predictions of penetration depth can be independent of the Δt value and thus reflect the objective essence of loading process. By taking Δt value as 0.001s for three materials with different constitutive properties, the P–h curves under load and displacement control manners are compared, and excellent agreement is found. The proposed method provides a new and elegant way to realize equivalent load control by displacement control of the indenter for nanoindentation simulations.
- Open Access
Collaborative Quantization Embeddings for Intra-Subject Prostate MR Image Registration
Image registration is useful for quantifying morphological changes in longitudinal MR images from prostate cancer patients. This paper describes a development in improving the learning-based registration algorithms, for this challenging clinical application often with highly variable yet limited training data. First, we report that the latent space can be clustered into a much lower dimensional space than that commonly found as bottleneck features at the deep layer of a trained registration network. Based on this observation, we propose a hierarchical quantization method, discretizing the learned feature vectors using a jointly-trained dictionary with a constrained size, in order to improve the generalisation of the registration networks. Furthermore, a novel collaborative dictionary is independently optimised to incorporate additional prior information, such as the segmentation of the gland or other regions of interest, in the latent quantized space. Based on 216 real clinical images from 86 prostate cancer patients, we show the efficacy of both the designed components. Improved registration accuracy was obtained with statistical significance, in terms of both Dice on gland and target registration error on corresponding landmarks, the latter of which achieved 5.46 mm, an improvement of 28.7\% from the baseline without quantization. Experimental results also show that the difference in performance was indeed minimised between training and testing data.
Open AccessCollaborative Quantization Embeddings for Intra-Subject Prostate MR Image Registration
Image registration is useful for quantifying morphological changes in longitudinal MR images from prostate cancer patients. This paper describes a development in improving the learning-based registration algorithms, for this challenging clinical application often with highly variable yet limited training data. First, we report that the latent space can be clustered into a much lower dimensional space than that commonly found as bottleneck features at the deep layer of a trained registration network. Based on this observation, we propose a hierarchical quantization method, discretizing the learned feature vectors using a jointly-trained dictionary with a constrained size, in order to improve the generalisation of the registration networks. Furthermore, a novel collaborative dictionary is independently optimised to incorporate additional prior information, such as the segmentation of the gland or other regions of interest, in the latent quantized space. Based on 216 real clinical images from 86 prostate cancer patients, we show the efficacy of both the designed components. Improved registration accuracy was obtained with statistical significance, in terms of both Dice on gland and target registration error on corresponding landmarks, the latter of which achieved 5.46 mm, an improvement of 28.7\% from the baseline without quantization. Experimental results also show that the difference in performance was indeed minimised between training and testing data.
- 13 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 23 (7) , 9529-9542
Confidence-and-Refinement Adaptation Model for Cross-Domain Semantic Segmentation
With the rapid development of convolutional neural networks (CNNs), significant progress has been achieved in semantic segmentation. Despite the great success, such deep learning approaches require large scale real-world datasets with pixel-level annotations. However, considering that pixel-level labeling of semantics is extremely laborious, many researchers turn to utilize synthetic data with free annotations. But due to the clear domain gap, the segmentation model trained with the synthetic images tends to perform poorly on the real-world datasets. Unsupervised domain adaptation (UDA) for semantic segmentation recently gains an increasing research attention, which aims at alleviating the domain discrepancy. Existing methods in this scope either simply align features or the outputs across the source and target domains or have to deal with the complex image processing and post-processing problems. In this work, we propose a novel multi-level UDA model named Confidence-and-Refinement Adaptation Model (CRAM), which contains a confidence-aware entropy alignment (CEA) module and a style feature alignment (SFA) module. Through CEA, the adaptation is done locally via adversarial learning in the output space, making the segmentation model pay attention to the high-confident predictions. Furthermore, to enhance the model transfer in the shallow feature space, the SFA module is applied to minimize the appearance gap across domains. Experiments on two challenging UDA benchmarks “GTA5-to-Cityscapes” and “SYNTHIA-to-Cityscapes” demonstrate the effectiveness of CRAM. We achieve comparable performance with the existing state-of-the-art works with advantages in simplicity and convergence speed.
- 13 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 23 (7) , 9529-9542
Confidence-and-Refinement Adaptation Model for Cross-Domain Semantic Segmentation
With the rapid development of convolutional neural networks (CNNs), significant progress has been achieved in semantic segmentation. Despite the great success, such deep learning approaches require large scale real-world datasets with pixel-level annotations. However, considering that pixel-level labeling of semantics is extremely laborious, many researchers turn to utilize synthetic data with free annotations. But due to the clear domain gap, the segmentation model trained with the synthetic images tends to perform poorly on the real-world datasets. Unsupervised domain adaptation (UDA) for semantic segmentation recently gains an increasing research attention, which aims at alleviating the domain discrepancy. Existing methods in this scope either simply align features or the outputs across the source and target domains or have to deal with the complex image processing and post-processing problems. In this work, we propose a novel multi-level UDA model named Confidence-and-Refinement Adaptation Model (CRAM), which contains a confidence-aware entropy alignment (CEA) module and a style feature alignment (SFA) module. Through CEA, the adaptation is done locally via adversarial learning in the output space, making the segmentation model pay attention to the high-confident predictions. Furthermore, to enhance the model transfer in the shallow feature space, the SFA module is applied to minimize the appearance gap across domains. Experiments on two challenging UDA benchmarks “GTA5-to-Cityscapes” and “SYNTHIA-to-Cityscapes” demonstrate the effectiveness of CRAM. We achieve comparable performance with the existing state-of-the-art works with advantages in simplicity and convergence speed.
- Top Cited Papers
- 9 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 40 (3) , 996-1006
Modeling and Enhancing Low-Quality Retinal Fundus Images
Retinal fundus images are widely used for the clinical screening and diagnosis of eye diseases. However, fundus images captured by operators with various levels of experience have a large variation in quality. Low-quality fundus images increase uncertainty in clinical observation and lead to the risk of misdiagnosis. However, due to the special optical beam of fundus imaging and structure of the retina, natural image enhancement methods cannot be utilized directly to address this. In this paper, we first analyze the ophthalmoscope imaging system and simulate a reliable degradation of major inferior-quality factors, including uneven illumination, image blurring, and artifacts. Then, based on the degradation model, a clinically oriented fundus enhancement network (cofe-Net) is proposed to suppress global degradation factors, while simultaneously preserving anatomical retinal structures and pathological characteristics for clinical observation and analysis. Experiments on both synthetic and real images demonstrate that our algorithm effectively corrects low-quality fundus images without losing retinal details. Moreover, we also show that the fundus correction method can benefit medical image analysis applications, e.g., retinal vessel segmentation and optic disc/cup detection.
- 9 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 40 (3) , 996-1006
Top Cited PapersModeling and Enhancing Low-Quality Retinal Fundus Images
Retinal fundus images are widely used for the clinical screening and diagnosis of eye diseases. However, fundus images captured by operators with various levels of experience have a large variation in quality. Low-quality fundus images increase uncertainty in clinical observation and lead to the risk of misdiagnosis. However, due to the special optical beam of fundus imaging and structure of the retina, natural image enhancement methods cannot be utilized directly to address this. In this paper, we first analyze the ophthalmoscope imaging system and simulate a reliable degradation of major inferior-quality factors, including uneven illumination, image blurring, and artifacts. Then, based on the degradation model, a clinically oriented fundus enhancement network (cofe-Net) is proposed to suppress global degradation factors, while simultaneously preserving anatomical retinal structures and pathological characteristics for clinical observation and analysis. Experiments on both synthetic and real images demonstrate that our algorithm effectively corrects low-quality fundus images without losing retinal details. Moreover, we also show that the fundus correction method can benefit medical image analysis applications, e.g., retinal vessel segmentation and optic disc/cup detection.
- 19 March 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 30 (12) , 4887-4898
Motion-Aware Rapid Video Saliency Detection
In this paper, we propose a computationally efficient and consistently accurate spatiotemporal salient object detection method to identify the most noticeable object in a video sequence. Intuitively, the underlying motion in a video is a more stable saliency indicator than the apparent color cues that often contain significant variations and complex structures. Based on this observation, we build an efficient and accurate spatiotemporal saliency detection method that uses motion information as a leverage to locate the most dynamic regions in a video sequence. We first analyze the optical flow field to obtain foreground priors, and then incorporate spatial saliency features such as appearance contrasts and compactness measures, into a multi-cue integration framework to combine various saliency cues and achieve temporal consistency. Rigorous experiments on the challenging SegTrackV1, SegTrackV2, and FBMS datasets demonstrate that our method generates comparable or superior performance to state-of-the-art methods while running almost 100× faster at only 0.08 sec/frame. Promising performance and rapid speed imply that the proposed spatiotemporal saliency method can be easily involved in various vision applications.
- 19 March 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 30 (12) , 4887-4898
Motion-Aware Rapid Video Saliency Detection
In this paper, we propose a computationally efficient and consistently accurate spatiotemporal salient object detection method to identify the most noticeable object in a video sequence. Intuitively, the underlying motion in a video is a more stable saliency indicator than the apparent color cues that often contain significant variations and complex structures. Based on this observation, we build an efficient and accurate spatiotemporal saliency detection method that uses motion information as a leverage to locate the most dynamic regions in a video sequence. We first analyze the optical flow field to obtain foreground priors, and then incorporate spatial saliency features such as appearance contrasts and compactness measures, into a multi-cue integration framework to combine various saliency cues and achieve temporal consistency. Rigorous experiments on the challenging SegTrackV1, SegTrackV2, and FBMS datasets demonstrate that our method generates comparable or superior performance to state-of-the-art methods while running almost 100× faster at only 0.08 sec/frame. Promising performance and rapid speed imply that the proposed spatiotemporal saliency method can be easily involved in various vision applications.