Hamed Sari-Sarraf

Significance: Circulating tumor cells (CTCs) are important biomarkers for cancer management. Isolated CTCs from blood are stained to detect and enumerate CTCs. However, the staining process is laborious and moreover makes CTCs unsuitable for drug testing and molecular characterization. Aim: The goal is to develop and test deep learning (DL) approaches to detect unstained breast cancer cells in bright-field microscopy images that contain white blood cells (WBCs). Approach: We tested two convolutional neural network (CNN) approaches. The first approach allows investigation of the prominent features extracted by CNN to discriminate in vitro cancer cells from WBCs. The second approach is based on faster region-based convolutional neural network (Faster R-CNN). Results: Both approaches detected cancer cells with higher than 95% sensitivity and 99% specificity with the Faster R-CNN being more efficient and suitable for deployment presenting an improvement of 4% in sensitivity. The distinctive feature that CNN uses for discrimination is cell size, however, in the absence of size difference, the CNN was found to be capable of learning other features. The Faster R-CNN was found to be robust with respect to intensity and contrast image transformations. Conclusions: CNN-based DL approaches could be potentially applied to detect patient-derived CTCs from images of blood samples.

Currently, detection of circulating tumor cells (CTCs) in cancer patient blood samples relies on immunostaining, which does not provide access to live CTCs, limiting the breadth of CTC-based applications. As a first step to address this limitation, here, we demonstrate staining-free enumeration of tumor cells spiked into lysed blood samples using digital holographic microscopy (DHM), microfluidics and machine learning (ML). A 3D-printed module for laser assembly was developed to simplify the optical set up for holographic imaging of cells flowing through a sheath-based microfluidic device. Computational reconstruction of the holograms was performed to localize the cells in 3D and obtain the plane of best focus images to train deep learning models. First, we evaluated the classification performance of two convolutional neural networks (CNNs): ResNet-50 and a custom-designed shallow Network dubbed s-Net. The accuracy, sensitivity and specificity of these networks were found to range from 97.08% and 99.32%. Upon selecting the s-Net due to its simple architecture and low computational burden, we formulated a decision gating strategy to significantly lower the false positive rate (FPR). By applying an optimized decision threshold to mixed samples prepared in silico , the FPR was reduced from 1×10 ⁻² to 2.77×10 ⁻⁴ . Finally, the developed DHM-ML framework was successfully applied to enumerate spiked MCF-7 breast cancer cells from lysed blood samples containing a background of white blood cells (WBCs). We conclude by discussing the advances that need to be made to translate the DHM-ML approach to staining-free enumeration of CTCs in cancer patient blood samples.

Significance Circulating tumor cells (CTCs) are important biomarkers for cancer management. Isolated CTCs from blood are stained to detect and enumerate CTCs. However, the staining process is laborious and moreover makes CTCs unsuitable for drug testing and molecular characterization. Aim The goal is to develop and test deep learning (DL) approaches to detect unstained breast cancer cells in bright field microscopy images that contain white blood cells (WBCs). Approach We tested two convolutional neural network (CNN) approaches. The first approach allows investigation of the prominent features extracted by CNN to discriminate cancer cells from WBCs. The second approach is based on Faster Region-based Convolutional Neural Network (Faster R-CNN). Results Both approaches detected cancer cells with high sensitivity and specificity with the Faster R-CNN being more efficient and suitable for deployment. The distinctive feature used by the CNN used to discriminate is cell size, however in the absence of size difference, the CNN was found to be capable of learning other features. The Faster R-CNN was found to be robust with respect to intensity and contrast image transformations. Conclusions CNN-based deep learning approaches could be potentially applied to detect patient-derived CTCs from images of blood samples.

This paper introduces an automated vision system for animal detection in trail-camera images taken from a field under the administration of the Texas Parks and Wildlife Department. As traditional wildlife counting techniques are intrusive and labor intensive to conduct, trail-camera imaging is a comparatively non-intrusive method for capturing wildlife activity. However, given the large volume of images produced from trail-cameras, manual analysis of the images remains time-consuming and inefficient. We implemented a two-stage deep convolutional neural network pipeline to find animal-containing images in the first stage and then process these images to detect birds in the second stage. The animal classification system classifies animal images with overall 93% sensitivity and 96% specificity. The bird detection system achieves better than 93% sensitivity, 92% specificity, and 68% average Intersection-over-Union rate. The entire pipeline processes an image in less than 0.5 seconds as opposed to an average 30 seconds for a human labeler. We also addressed post-deployment issues related to data drift for the animal classification system as image features vary with seasonal changes. This system utilizes an automatic retraining algorithm to detect data drift and update the system. We introduce a novel technique for detecting drifted images and triggering the retraining procedure. Two statistical experiments are also presented to explain the prediction behavior of the animal classification system. These experiments investigate the cues that steers the system towards a particular decision. Statistical hypothesis testing demonstrates that the presence of an animal in the input image significantly contributes to the system's decisions.

A prior work describing a computer vision system for measuring maturity using longitudinal views of cotton fibers reported observing a large variation in maturity within a single fiber. This paper describes the use of confocal microscopy as an independent measuring method to validate those findings. Individual cotton fibers are imaged from end to end by a confocal microscope producing hundreds of image volumes of cotton fiber segments, each ∼150 µm in length. From these volumes, virtual fiber cross-sections are extracted, processed using the level set method, and measured according to AATCC standards. The results demonstrate with both visual and quantitative analysis that fiber maturity can exhibit large variations within a single fiber.

We introduce a novel Maximum Entropy (MaxEnt) framework that can generate 3D scenes by incorporating objects’ relevancy, hierarchical and contextual constraints in a unified model. This model is formulated by a Gibbs distribution, under the MaxEnt framework, that can be sampled to generate plausible scenes. Unlike existing approaches, which represent a given scene by a single And-Or graph, the relevancy constraint (defined as the frequency with which a given object exists in the training data) require our approach to sample from multiple And-Or graphs, allowing variability in terms of objects’ existence across synthesized scenes. Once an And-Or graph is sampled from the ensemble, the hierarchical constraints are employed to sample the Or-nodes (style variations) and the contextual constraints are subsequently used to enforce the corresponding relations that must be satisfied by the And-nodes. To illustrate the proposed methodology, we use desk scenes that are composed of objects whose existence, styles and arrangements (position and orientation) can vary from one scene to the next. The relevancy, hierarchical and contextual constraints are extracted from a set of training scenes and utilized to generate plausible synthetic scenes that in turn satisfy these constraints. After applying the proposed framework, scenes that are plausible representations of the training examples are automatically generated.