H-EMD: A Hierarchical Earth Mover’s Distance Method for Instance Segmentation
- 21 April 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 41 (10) , 2582-2597
Abstract
Deep learning (DL) based semantic segmentation methods have achieved excellent performance in biomedical image segmentation, producing high quality probability maps to allow extraction of rich instance information to facilitate good instance segmentation. While numerous efforts were put into developing new DL semantic segmentation models, less attention was paid to a key issue of how to effectively explore their probability maps to attain the best possible instance segmentation. We observe that probability maps by DL semantic segmentation models can be used to generate many possible instance candidates, and accurate instance segmentation can be achieved by selecting from them a set of “optimized” candidates as output instances. Further, the generated instance candidates form a well-behaved hierarchical structure (a forest), which allows selecting instances in an optimized manner. Hence, we propose a novel framework, called hierarchical earth mover’s distance (H-EMD), for instance segmentation in biomedical 2D+time videos and 3D images, which judiciously incorporates consistent instance selection with semantic-segmentation-generated probability maps. H-EMD contains two main stages: (1) instance candidate generation: capturing instance-structured information in probability maps by generating many instance candidates in a forest structure; (2) instance candidate selection: selecting instances from the candidate set for final instance segmentation. We formulate a key instance selection problem on the instance candidate forest as an optimization problem based on the earth mover’s distance (EMD), and solve it by integer linear programming. Extensive experiments on eight biomedical video or 3D datasets demonstrate that H-EMD consistently boosts DL semantic segmentation models and is highly competitive with state-of-the-art methods.Keywords
All Related Versions
Funding Information
- NSF (CCF-1617735)
- U.S. Army Research Office (W911NF-17-1-0448)
- Notre Dame Eck Institute for Global Health
This publication has 56 references indexed in Scilit:
- Segmentation of Cells from Spinning Disk Confocal Images Using a Multi-stage Approach2015
- Graphical model for joint segmentation and tracking of multiple dividing cellsBioinformatics, 2014
- Global Linking of Cell Tracks Using the Viterbi AlgorithmIEEE Transactions on Medical Imaging, 2014
- A Comparative Review of Component Tree Computation AlgorithmsIEEE Transactions on Image Processing, 2014
- A benchmark for comparison of cell tracking algorithmsBioinformatics, 2014
- Vertebrate kidney tubules elongate using a planar cell polarity–dependent, rosette-based mechanism of convergent extensionNature Genetics, 2012
- Quasi-Linear Algorithms for the Topological WatershedJournal of Mathematical Imaging and Vision, 2005
- Mean shift: a robust approach toward feature space analysisIEEE Transactions on Pattern Analysis and Machine Intelligence, 2002
- Connected Filtering and Segmentation Using Component TreesComputer Vision and Image Understanding, 1999
- Topographic distance and watershed linesSignal Processing, 1994