STACT-Time: Spatio-Temporal Cross Attention for Cine Thyroid Ultrasound Time Series Classification
Authors:
Preprint
Abstract
Thyroid cancer is among the most common cancers in the United States. Thyroid nodules are frequently detected through ultrasound (US) imaging, and some require further evaluation via fine-needle aspiration (FNA) biopsy. Despite its effectiveness, FNA often leads to unnecessary biopsies of benign nodules, causing patient discomfort and anxiety. To address this, the American College of Radiology Thyroid Imaging Reporting and Data System (TI-RADS) has been developed to reduce benign biopsies. However, such systems are limited by interobserver variability. Recent deep learning approaches have sought to improve risk stratification, but they often fail to utilize the rich temporal and spatial context provided by US cine clips, which contain dynamic global information and surrounding structural changes across various views. In this work, we propose the Spatio-Temporal Cross Attention for Cine Thyroid Ultrasound Time Series Classification (STACT-Time) model, a novel representation learning framework that integrates imaging features from US cine clips with features from segmentation masks automatically generated by a pretrained model. By leveraging self-attention and cross-attention mechanisms, our model captures the rich temporal and spatial context of US cine clips while enhancing feature representation through segmentation-guided learning. Our model improves malignancy prediction compared to state-of-the-art models, achieving a cross-validation precision of 0.91 (plus or minus 0.02) and an F1 score of 0.89 (plus or minus 0.02). By reducing unnecessary biopsies of benign nodules while maintaining high sensitivity for malignancy detection, our model has the potential to enhance clinical decision-making and improve patient outcomes.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- An interpretable two-branch bi-coordinate network based on multi-grained domain knowledge for classification of thyroid nodules in ultrasound imagesMedical Image Analysis, 2024
- Human understandable thyroid ultrasound imaging AI report system — A bridge between AI and cliniciansiScience, 2023
- Toward Reduction in False-Positive Thyroid Nodule Biopsies with a Deep Learning–based Risk Stratification System Using US Cine-Clip ImagesRadiology: Artificial Intelligence, 2022
- MSU-Net: Multi-Scale U-Net for 2D Medical Image SegmentationFrontiers in Genetics, 2021
- Differential Diagnosis of Benign and Malignant Thyroid Nodules Using Deep Learning Radiomics of Thyroid Ultrasound ImagesEuropean Journal of Radiology, 2020
- Management of Thyroid Nodules Seen on US Images: Deep Learning May Match Performance of RadiologistsRadiology, 2019
- Reducing the Number of Unnecessary Thyroid Biopsies While Improving Diagnostic Accuracy: Toward the “Right” TIRADSThe Journal of Clinical Endocrinology & Metabolism, 2018
- ACR Thyroid Imaging, Reporting and Data System (TI-RADS): White Paper of the ACR TI-RADS CommitteeJournal of the American College of Radiology, 2017
- Managing Incidental Thyroid Nodules Detected on Imaging: White Paper of the ACR Incidental Thyroid Findings CommitteeJournal of the American College of Radiology, 2015
- AIUM Practice Guideline for the Performance of a Thyroid and Parathyroid Ultrasound ExaminationJournal of Ultrasound in Medicine, 2013