Developmental Stage Classification of Embryos Using Two-Stream Neural Network with Linear-Chain Conditional Random Field
- 21 September 2021
- book chapter
- Published by Springer Nature
- Vol. 12908, 363-372
Abstract
The developmental process of embryos follows a monotonic order. An embryo can progressively cleave from one cell to multiple cells and finally transform to morula and blastocyst. For time-lapse videos of embryos, most existing developmental stage classification methods conduct per-frame predictions using an image frame at each time step. However, classification using only images suffers from overlapping between cells and imbalance between stages. Temporal information can be valuable in addressing this problem by capturing movements between neighboring frames. In this work, we propose a two-stream model for developmental stage classification. Unlike previous methods, our two-stream model accepts both temporal and image information. We develop a linear-chain conditional random field (CRF) on top of neural network features extracted from the temporal and image streams to make use of both modalities. The linear-chain CRF formulation enables tractable training of global sequential models over multiple frames while also making it possible to inject monotonic development order constraints into the learning process explicitly. We demonstrate our algorithm on two time-lapse embryo video datasets: i) mouse and ii) human embryo datasets. Our method achieves 98.1% and 80.6% for mouse and human embryo stage classification, respectively. Our approach will enable more profound clinical and biological studies and suggests a new direction for developmental stage classification by utilizing temporal information.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Cell-Net: Embryonic Cell Counting and Centroid Localization via Residual Incremental Atrous Pyramid and Progressive Upsampling ConvolutionIEEE Access, 2019
- Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset2017
- Deep Convolutional Neural Networks for Human Embryonic Cell Counting2016
- Deep Residual Learning for Image Recognition2016
- Convolutional Two-Stream Network Fusion for Video Action Recognition2016
- Large-Scale Video Classification with Convolutional Neural Networks2014
- Label free cell-tracking and division detection based on 2D time-lapse images for lineage analysis of early embryo developmentComputers in Biology and Medicine, 2014
- ImageNet: A large-scale hierarchical image database2009
- Learning to forget: continual prediction with LSTM1999
- Determining optical flowArtificial Intelligence, 1981