Motion-Corrected Moving Average: Including Post-Hoc Temporal Information for Improved Video Segmentation
Preprint
Abstract
Real-time computational speed and a high degree of precision are requirements for computer-assisted interventions. Applying a segmentation network to a medical video processing task can introduce significant inter-frame prediction noise. Existing approaches can reduce inconsistencies by including temporal information but often impose requirements on the architecture or dataset. This paper proposes a method to include temporal information in any segmentation model and, thus, a technique to improve video segmentation performance without alterations during training or additional labeling. With Motion-Corrected Moving Average, we refine the exponential moving average between the current and previous predictions. Using optical flow to estimate the movement between consecutive frames, we can shift the prior term in the moving-average calculation to align with the geometry of the current frame. The optical flow calculation does not require the output of the model and can therefore be performed in parallel, leading to no significant runtime penalty for our approach. We evaluate our approach on two publicly available segmentation datasets and two proprietary endoscopic datasets and show improvements over a baseline approach.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Error-Correcting Mean-Teacher: Corrections instead of consistency-targets applied to semi-supervised medical image segmentationComputers in Biology and Medicine, 2023
- FUN-SIS: A Fully UNsupervised approach for Surgical Instrument SegmentationMedical Image Analysis, 2023
- DRR-Net: A Dense-Connected Residual Recurrent Convolutional Network for Surgical Instrument Segmentation From Endoscopic ImagesIEEE Transactions on Medical Robotics and Bionics, 2022
- Multi-Frame Feature Aggregation for Real-Time Instrument Segmentation in Endoscopic VideoIEEE Robotics and Automation Letters, 2021
- Comparative validation of multi-instance instrument segmentation in endoscopy: Results of the ROBUST-MIS 2019 challengeMedical Image Analysis, 2021
- ST-MTL: Spatio-Temporal multitask learning model to predict scanpath while tracking instruments in robotic surgeryMedical Image Analysis, 2021
- Accel: A Corrective Fusion Network for Efficient Semantic Segmentation on Video2019
- Deep Feature Flow for Video Recognition2017
- Fully Convolutional Networks for Semantic SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- SIFT Flow: Dense Correspondence across Scenes and Its ApplicationsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010