Guanqiao Shan

Quantitative analysis of cleavage-stage embryos is a critical step for embryo evaluation in in vitro fertilization (IVF) treatment. Deep learning-based instance segmentation methods have emerged as a promising solution. However, existing methods, typically designed for opaque instances in nature scenes, face challenges when applied to segmenting overlapping cells in cleavage-stage embryos. The semi-transparency of the cells in cleavage-stage embryos leads to vague boundaries and complex overlaps between cells, making it difficult for existing models to segment these cells without merging or missing cells. In this work, we propose a novel instance segmentation network for segmenting overlapping, semi-transparent cleavage-stage embryo cells. Specifically, the network introduces two specialized branches that learn complementary representations: an overlap branch for addressing inter-instance overlaps and a boundary branch for dealing with intra-instance shapes. Then, an attention-based feature fusion module is designed to integrate learned overlap and boundary features to enhance the instance representation for the segmentation branch. Finally, an overlap-aware post-processing module leverages the outputs from the overlap and boundary branches to adaptively adjust suppression thresholds, ensuring the preservation of distinct yet highly overlapping cells. Experiments on two cleavage-stage embryo datasets demonstrate that our method outperformed state-of-the-art methods in both segmentation accuracy and recall, highlighting its potential for quantitative embryo evaluation in clinical IVF.

Segmenting individual trophectoderm (TE) cells is essential for developing quantitative metrics to assess the developmental potential of human blastocysts. The elongated shape and circular arrangement of TE cells lead to continuously varying orientations across the image, posing challenges for existing cell instance segmentation methods that assume uniformly oriented cells. As a result, most methods segment the TE as a single region, and the development of quantitative, cell-level metrics predictive of live birth potential remains unexplored. In this work, we propose an instance segmentation model that represents elongate, circularly arranged TE cells using elliptical distance maps, with which superior performance in both segmentation accuracy and metric extraction was achieved, compared with state-of-the-art methods. The extracted metrics, including TE cell number, the mean and standard deviation of cell length, width, area, and mean inter-cell distance, serve as effective predictors of a blastocyst’s live birth potential. When used to predict live birth, these metrics achieved a significantly higher area under the receiver operating characteristic curve (AUC = 0.693) than traditional TE morphological grades (AUC = 0.585). The source code is publicly available at https://github.com/robotVisionHang/TESeg.

Sperm DNA integrity indicates genetic quality and stability, which is vital for successful fertilization, embryo development, and the overall health of offspring. Currently, DNA integrity is assessed manually by clinical staff through invasive staining procedures, which are labor-intensive and alter sperm viability. In this work, we propose a deep learning-based approach to automatically predict DNA integrity in live sperm without manual operation or invasive staining. Specifically, the proposed method performs pixel-wise prediction of ssDNA and dsDNA for accurate DNA Fragmentation Index (DFI) calculation. A coupling network is introduced to improve both prediction accuracy and interpretability. To effectively capture both local and global features, the enhancement model integrates convolutional operations into the Transformer architecture. Additionally, a cross-scale fusion Transformer block is designed to fuse multi-scale information using a cross-attention mechanism. To further enhance prediction accuracy, a high-frequency branch is incorporated to guide the network's focus toward fine-grained, high-frequency details. Experimental results on a sperm DFI dataset demonstrate that the proposed method outperforms existing enhancement networks.

Automated morphological measurement of cellular and subcellular structures in live cells is important for evaluating cell functions. Due to their small size and transparent appearance, visualizing cellular and subcellular structures often requires high magnification microscopy and fluorescent staining. However, high magnification microscopy gives a limited field of view, and fluorescent staining alters cell viability and/or activity. Therefore, microscopy image enhancement methods have been developed to predict detailed intracellular structures in live cells. Existing image enhancement networks are mostly CNN-based models lacking global information or Transformer-based models lacking local information. For these purposes, a novel CNN-Transformer combined bilateral U-Net (CTBUnet) is proposed to effectively aggregate both local and global information. Experiments on the collected sperm cell enhancement dataset demonstrate the effectiveness of proposed network for both super-resolution and virtual staining prediction.

Sperm morphology measurement is vital for diagnosing male infertility, which involves quantification of multiple subcellular parts for each sperm. Instance-aware part segmentation networks have been introduced to address this task by automatically identifying individual sperm and segmenting their subcellular parts. However, major limitations of state-of-the-art instance-aware part segmentation networks include: 1) they are time-consuming and computational expensive due to sequential processing and multi-stage frameworks; 2) they perform poorly for densely packed sperm that overlap or cross over one another. To overcome these challenges, this paper proposes 1) integrating instance identification and subcellular part segmentation within a single-stage framework to save inference time and memory usage; 2) dividing a sperm target into simpler components (head and tail) to improve prediction accuracy, followed by a contrastive learning-based matching method to pair the head and tail. Experimental results on our clinically collected human sperm dataset demonstrated that the proposed network not only outperformed state-of-the-art CP-Net (by 3.5% APp vol) but also achieved realtime inference (48.0 frames per second), effectively meeting the clinical requirements for automated parts segmentation of sperm. final part segmentation results. 2) Since the sperm head and tail have simpler shapes, they are detected separately to improve segmentation accuracy. A contrastive learning-based method is then designed to pair head and tail based on similarity of feature embeddings extracted from the proposed instance prediction branch. The proposed method significantly outperformed existing networks, particularly in handling densely packed sperm. The presented method has applicability to analyzing sperm and more broadly other cell types.

STUDY QUESTION: Can a quantitative method be developed to differentiate between blastocysts with similar or same inner cell mass (ICM) and trophectoderm (TE) grades, while also reflecting their potential for live birth? SUMMARY ANSWER: We developed BlastScoringNet, an interpretable deep-learning model that quantifies blastocyst ICM and TE morphology with continuous scores, enabling finer differentiation between blastocysts with similar or same grades, with higher scores significantly correlating with higher live birth rates. WHAT IS KNOWN ALREADY: While the Gardner grading system is widely used by embryologists worldwide, blastocysts having similar or same ICM and TE grades cause challenges for embryologists in decision-making. Furthermore, human assessment is subjective and inconsistent in predicting which blastocysts have higher potential to result in live birth. STUDY DESIGN, SIZE, DURATION: The study design consists of three main steps. First, BlastScoringNet was developed using a grading dataset of 2760 blastocysts with majority-voted Gardner grades. Second, the model was applied to a live birth dataset of 15 228 blastocysts with known live birth outcomes to generate blastocyst scores. Finally, the correlation between these scores and live birth outcomes was assessed. The blastocysts were collected from patients who underwent IVF treatments between 2016 and 2018. For external application study, an additional grading dataset of 1455 blastocysts and a live birth dataset of 476 blastocysts were collected from patients who underwent IVF treatments between 2021 and 2023 at an external IVF institution. PARTICIPANTS/MATERIALS, SETTING, METHODS: In this retrospective study, we developed BlastScoringNet, an interpretable deep-learning model which outputs expansion degree grade and continuous scores quantifying a blastocyst’s ICM morphology and TE morphology, based on the Gardner grading system. The continuous ICM and TE scores were calculated by weighting each base grade’s predicted probability and summing the predicted probabilities. To represent each blastocyst’s overall potential for live birth, we combined the ICM and TE scores using their odds ratios (ORs) for live birth. We further assessed the correlation between live birth rates and the ICM score, TE score, and the OR-combined score (adjusted for expansion degree) by applying BlastScoringNet to blastocysts with known live birth outcomes. To test its generalizability, we also applied BlastScoringNet to an external IVF institution, accounting for variations in imaging conditions, live birth rates, and embryologists’ experience levels. MAIN RESULTS AND THE ROLE OF CHANCE: BlastScoringNet was developed using data from 2760 blastocysts with majority-voted grades for expansion degree, ICM, and TE. The model achieved mean area under the receiver operating characteristic curve values of 0.997 (SD 0.004) for expansion degree, 0.903 (SD 0.031) for ICM, and 0.943 (SD 0.040) for TE, based on predicted probabilities for each base grade. From these predicted probabilities, BlastScoringNet generated continuous ICM and TE scores, as well as expansion degree grades, for an additional 15 228 blastocysts with known live birth outcomes. Higher ICM and TE scores, along with their OR-combined scores, were significantly correlated with increased live birth rates (P < 0.0001). By fine-tuning, BlastScoringNet was applied to an external IVF institution, where higher OR-combined ICM and TE scores also significantly correlated with increased live birth rates (P = 0.00078), demonstrating consistent results across both institutions. LIMITATIONS, REASONS FOR CAUTION: This study is limited by its retrospective nature. Further prospective randomized trials are required to confirm the clinical impact of BlastScoringNet in assisting embryologists in blastocyst selection. WIDER IMPLICATIONS OF THE FINDINGS: BlastScoringNet provides an interpretable and quantitative method for evaluating blastocysts, aligned with the widely used Gardner grading system. Higher OR-combined ICM and TE scores, representing each blastocyst’s overall potential for live birth, were significantly correlated with increased live birth rates. The model’s demonstrated generalizability across two institutions further supports its clinical utility. These findings suggest that BlastScoringNet is a valuable tool for assisting embryologists in selecting blastocysts with the highest potential for live birth. The code and pre-trained models are publicly available to facilitate further research and widespread implementation. STUDY FUNDING/COMPETING INTEREST(S): This work was supported by the Vector Institute and the Temerty Faculty of Medicine at the University of Toronto, Toronto, Ontario, Canada, via a Clinical AI Integration Grant, and the Natural Science Foundation of Hunan Province of China (2023JJ30714). The authors declare no competing interests. TRIAL REGISTRATION NUMBER: N/A.

Automated positioning systems play a pivotal role in micro-scale cell manipulation. In clinical intracytoplasmic sperm injection (ICSI) for infertility treatment, a motile sperm needs to be immobilized by glass micropipette tapping for subsequent surgical steps. The process requires accurate tracking of the target sperm and precise alignment between the sperm tail and the micropipette. Manual sperm immobilization suffers from inconsistent success rates, and current robotic systems developed for the task fail to comply with the standard clinical setup. Instead of using a motorized micromanipulator as in existing robotic systems, this paper presents an automated and compact three-dimensional (3-D) positioning stage for sperm immobilization that can be seamlessly integrated into standard clinical platforms. To tackle the challenge of accurately tracking the target sperm with degraded detection quality due to the complex 3-D motion of the positioning stage, a multi-stage sperm tracking scheme is designed for detection-to-tracklet association. To prevent physical contact between the sperm head and the micropipette, an adaptive tail-tapping planning strategy based on the sperm head orientation analysis is established. A visual servo controller equipped with a dynamic sperm motion observer is further employed to achieve precise positioning of the target sperm during the immobilization process. Experimental results demonstrated that the proposed system achieved a sperm tracking accuracy of 88.12%, and a sperm positioning accuracy of 2.3 ± 1.2 μm. Further experiments revealed the system achieved a success rate of 93.5% and a time cost of 5.5 s for automated sperm immobilization.

Arrhythmogenic cardiomyopathy (ACM) is a leading cause of sudden cardiac death among young adults. Aberrant gap junction remodeling has been linked to disease-causative mutations in plakophilin-2 ( PKP2 ). Although gap junctions are a key therapeutic target, measurement of gap junction function in preclinical disease models is technically challenging. To quantify gap junction function with high precision and high consistency, we developed a robotic cell manipulation system with visual feedback from digital holographic microscopy for three-dimensional and label-free imaging of human induced pluripotent stem cell–derived cardiomyocytes (iPSC-CMs). The robotic system can accurately determine the dynamic height changes in the cells’ contraction and resting phases, microinject drug-treated healthy and diseased iPSC-CMs in their resting phase with constant injection depth across all cells, and deposit a membrane-impermeable dye that solely diffuses between cells through gap junctions for measuring the gap junction diffusion function. The robotic system was applied toward a targeted drug screen to identify gap junction modulators and potential therapeutics for ACM. Five compounds were found to dose-dependently enhance gap junction permeability in cardiomyocytes with PKP2 knockdown. In addition, PCO 400 (pinacidil) reduced beating irregularity in a mouse model of ACM expressing mutant PKP2 (R735X). These results highlight the utility of the robotic cell manipulation system to efficiently assess gap junction function in a relevant preclinical disease model, thus providing a technique to advance drug discovery for ACM and other gap junction–mediated diseases.