Dayong Wang

Metastatic breast cancer is one of the leading causes of cancer mortality. While there has been progress in developing deep learning-driven diagnostic system for metastatic breast cancer based on histopathological images, it faces a major challenge in real-world application, i.e., how to improve generalizability of the diagnostic models for diverse patient populations and variations in sample and image processing across different geographic regions. Multi-institutional collaborative learning, which trains a single model using data from multiple institutions, can address the challenges of data inadequacy, lack of data diversity, and data quality. However, the current practice of direct medical data transfer among institutions faces increasing restrictions including patient privacy, intellectual property, data ownership and legal obligations. To enable multi-institutional collaborative learning for cancer diagnosis, we developed a federated learning-driven collaborative diagnostic system for metastatic breast cancer. This system preserves patient privacy by decoupling model training from the need for direct medical data transfer between institutions. Further, this study has demonstrated that the federate learning-driven system can improve diagnostic model accuracy and generalizability by leveraging information derived from diverse data sources across multiple institutions. This study has also shown that this collaborative diagnostic system has a strong potential of improving local diagnosis performance on lower quality images at a resource constrained institution. Our research indicated that federated learning-driven diagnostic system is a feasible and robust framework for multi-institutional collaborative development of diagnostic models for metastatic breast cancer. In addition to the benefits of improving diagnostic model generalizability for diverse patient populations, the collaborative diagnostic system presents a new opportunity to enable under-resourced healthcare institutions to leverage external data resources to improve local diagnosis performance while preserving patient privacy.

Breast cancer is one of the leading causes of cancer mortality. Breast cancer patients in developing countries, especially sub-Saharan Africa, South Asia, and South America, suffer from the highest mortality rate in the world. One crucial factor contributing to the global disparity in mortality rate is long delay of diagnosis due to a severe shortage of trained pathologists, which consequently has led to a large proportion of late-stage presentation at diagnosis. To tackle this critical healthcare disparity, we have developed a deep learning-based diagnosis system for metastatic breast cancer that can achieve high diagnostic accuracy as well as computational efficiency and mobile readiness suitable for an under-resourced environment. We evaluated 4 Convolutional Neural Network (CNN) architectures: MobileNetV2, VGG16, ResNet50 and ResNet101. The MobileNetV2-based diagnostic model outperformed the more complex VGG16, ResNet50 and ResNet101 models in diagnostic accuracy, model generalization, and model training efficiency. The ROC AUC of MobilenetV2 (0.933, 95% CI: 0.930, 0.936) was higher than VGG16 (0.911, 95% CI: 0.908, 0.915), ResNet50 (0.869, 95% CI: 0.866, 0.873), and ResNet101 (0.873, 95% CI: 0.869, 0.876). The time per inference step for the MobileNetV2 model (15 ms/step) was substantially lower than that of VGG16 (48 ms/step), ResNet50 (37 ms/step), and ResNet110 (56 ms/step). The visual comparisons between the model prediction and ground truth have demonstrated that the MobileNetV2 diagnostic models can identify very small cancerous nodes embedded in a large area of normal cells which is challenging for manual image analysis. Equally Important, the light weight MobleNetV2 models were computationally efficient and ready for mobile devices or devices of low computational power. These advances empower the development of a resource-efficient and high performing AI-based metastatic breast cancer diagnostic system that can adapt to under-resourced healthcare facilities in developing countries.

Quiz Ref ID Full digitalization of the microscopic evaluation of stained tissue sections in histopathology has become feasible in recent years because of advances in slide scanning technology and cost reduction in digital storage. Advantages of digital pathology include remote diagnostics, immediate availability of archival cases, and easier consultations with expert pathologists.¹ Also, the possibility for computer-aided diagnostics may be advantageous.²

Current analysis of tumor proliferation, the most salient breast cancer prognostic biomarker, is limited to subjective mitosis counting by pathologists in localized regions of tissue images. This study presents the first data-driven integrative approach to characterize the severity of tumor growth and spread on a categorical and molecular level, utilizing multiple biologically salient deep learning classifiers to develop a comprehensive prognostic model. Our approach achieves pathologist-level performance on three-class categorical tumor severity prediction. It additionally pioneers prediction of molecular expression data from a tissue image, obtaining a Spearman's rank correlation coefficient of 0.60 with ex vivo mean calculated RNA expression. Furthermore, our framework is applied to identify over two hundred unprecedented biomarkers critical to the accurate assessment of tumor proliferation, validating our proposed integrative pipeline as the first to holistically and objectively analyze histopathological images.

Given a large collection of unlabeled face images, we address the problem of clustering faces into an unknown number of identities. This problem is of interest in social media, law enforcement, and other applications, where the number of faces can be of the order of hundreds of million, while the number of identities (clusters) can range from a few thousand to millions. To address the challenges of run-time complexity and cluster quality, we present an approximate Rank-Order clustering algorithm that performs better than popular clustering algorithms (k-Means and Spectral). Our experiments include clustering up to 123 million face images into over 10 million clusters. Clustering results are analyzed in terms of external (known face labels) and internal (unknown face labels) quality measures, and run-time. Our algorithm achieves an F-measure of 0.87 on the LFW benchmark (13 K faces of 5,749 individuals), which drops to 0.27 on the largest dataset considered (13 K faces in LFW + 123M distractor images). Additionally, we show that frames in the YouTube benchmark can be clustered with an F-measure of 0.71. An internal per-cluster quality measure is developed to rank individual clusters for manual exploration of high quality clusters that are compact and isolated.

Given the prevalence of social media websites, one challenge facing computer vision researchers is to devise methods to search for persons of interest among the billions of shared photos on these websites. Despite significant progress in face recognition, searching a large collection of unconstrained face images remains a difficult problem. To address this challenge, we propose a face search system which combines a fast search procedure, coupled with a state-of-the-art commercial off the shelf (COTS) matcher, in a cascaded framework. Given a probe face, we first filter the large gallery of photos to find the top-k most similar faces using features learned by a convolutional neural network. The k retrieved candidates are re-ranked by combining similarities based on deep features and those output by the COTS matcher. We evaluate the proposed face search system on a gallery containing 80 million web-downloaded face images. Experimental results demonstrate that while the deep features perform worse than the COTS matcher on a mugshot dataset (93.7 percent versus 98.6 percent TAR@FAR of 0.01 percent), fusing the deep features with the COTS matcher improves the overall performance (99.5 percent TAR@FAR of 0.01 percent). This shows that the learned deep features provide complementary information over representations used in state-of-the-art face matchers. On the unconstrained face image benchmarks, the performance of the learned deep features is competitive with reported accuracies. LFW database: 98.20 percent accuracy under the standard protocol and 88.03 percent TAR@FAR of 0.1 percent under the BLUFR protocol; IJB-A benchmark: 51.0 percent TAR@FAR of 0.1 percent (verification), rank 1 retrieval of 82.2 percent (closed-set search), 61.5 percent FNIR@FAR of 1 percent (open-set search). The proposed face search system offers an excellent trade-off between accuracy and scalability on galleries with millions of images. Additionally, in a face search experiment involving photos of the Tsarnaev brothers, convicted of the Boston Marathon bombing, the proposed cascade face search system could find the younger brother's (Dzhokhar Tsarnaev) photo at rank 1 in 1 second on a 5 M gallery and at rank 8 in 7 seconds on an 80 M gallery.

Face retrieval is an enabling technology for many applications, including automatic face annotation, deduplication, and surveillance. In this paper, we propose a face retrieval system which combines a k-NN search procedure with a COTS matcher (PittPatt ¹ ) in a cascaded manner. In particular, given a query face, we first pre-filter the gallery set and find the top-k most similar faces for the query image by using deep facial features that are learned with a deep convolutional neural network. The top-k most similar faces are then re-ranked based on score-level fusion of the similarities between deep features and the COTS matcher. To further boost the retrieval performance, we develop a manifold ranking algorithm. The proposed face retrieval system is evaluated on two large-scale face image databases: (i) a web face image database, which consists of over 3, 880 query images of 1, 507 subjects and a gallery of 5, 000, 000 faces, and (ii) a mugshot database, which consists of 1, 000 query images of 1, 000 subjects and a gallery of 1, 000, 000 faces. Experimental results demonstrate that the proposed face retrieval system can simultaneously improve the retrieval performance (CMC and precision-recall) and scalability for large-scale face retrieval problems.

The amount of data in our society has been exploding in the era of big data today. In this paper, we address several open challenges of big data stream classification, including high volume, high velocity, high dimensionality, and high sparsity. Many existing studies in data mining literature solve data stream classification tasks in a batch learning setting, which suffers from poor efficiency and scalability when dealing with big data. To overcome the limitations, this paper investigates an online learning framework for big data stream classification tasks. Unlike some existing online data stream classification techniques that are often based on first-order online learning, we propose a framework of Sparse Online Classification (SOC) for data stream classification, which includes some state-of-the-art first-order sparse online learning algorithms as special cases and allows us to derive a new effective second-order online learning algorithm for data stream classification. We conduct an extensive set of experiments, in which encouraging results validate the efficacy of the proposed algorithms in comparison to a family of state-of-the-art techniques on a variety of data stream classification tasks.

Elastic network models (ENM) are based on the idea that the geometry of a protein structure provides enough information for computing its fluctuations around its equilibrium conformation. This geometry is represented as an elastic network (EN) that is, a network of links between residues. A spring is associated with each of these links. The normal modes of the protein are then identified with the normal modes of the corresponding network of springs. Standard approaches for generating ENs rely on a cutoff distance. There is no consensus on how to choose this cutoff. In this work, we propose instead to filter the set of all residue pairs in a protein using the concept of alpha shapes. The main alpha shape we considered is based on the Delaunay triangulation of the Cα positions; we referred to the corresponding EN as EN(∞). We have shown that heterogeneous anisotropic network models, called αHANMs, that are based on EN(∞) reproduce experimental B-factors very well, with correlation coefficients above 0.99 and root-mean-square deviations below 0.1 Ų for a large set of high resolution protein structures. The construction of EN(∞) is simple to implement and may be used automatically for generating ENs for all types of ENMs.