Abstract
Locating the start, apex and end keyframes of moving contrast agents for keyframe counting in X-ray coronary angiography (XCA) is very important for the diagnosis and treatment of cardiovascular diseases. To locate these keyframes from the class-imbalanced and boundary-agnostic foreground vessel actions that overlap complex backgrounds, we propose long short-term spatiotemporal attention by integrating a convolutional long short-term memory (CLSTM) network into a multiscale Transformer to learn the segment-and sequence-level dependencies in the consecutive-frame-based deep features. Image-to-patch contrastive learning is further embedded between the CLSTM-based long-term spatiotemporal attention and Transformer-based short-term attention modules. The imagewise contrastive module reuses the long-term attention to contrast image-level foreground/background of XCA sequence, while patchwise contrastive projection selects the random patches of backgrounds as convolution kernels to project foreground/background frames into different latent spaces. A new XCA video dataset is collected to evaluate the proposed method. The experimental results show that the proposed method achieves a mAP (mean average precision) of 72.45% and a F-score of 0.8296, considerably outperforming the state-of-the-art methods. The source code and dataset are available at https://github.com/Binjie-Qin/STA-IPCon.
Funding Information
  • Science and Technology Commission of Shanghai Municipality (19dz1200500, 19411951507)
  • National Natural Science Foundation of China (61271320, 82070477)
  • Shanghai Science and Technology Committee (22XD1421800)
  • Shanghai “Rising Stars of Medical Talent” Youth Development Program “Outstanding Youth Medical Talents” (SHWSRS (2021) 099)
  • New Interdisciplinary Research Project of Shanghai Municipal Health Commission (2022JC022)
  • Shanghai Jiao Tong University Medical-Engineering Cross Research Fund (ZH2018ZDA19, YG2021QN122, YG2021QN99)

This publication has 72 references indexed in Scilit: