Hae Yong Kim
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Hae Yong Kim has a total of 42 co-authors. Here are the top co-authors by number of shared publications:
- Daniel G. P. Petrini
Escola Politecnica da Universidade de Sao Paulo, Sao Paulo, Brazil
4 shared publications - Guilherme Apolinario Silva Novaes
Escola Politecnica da Universidade de Sao Paulo, Sao Paulo, Brazil
3 shared publications Rosimeire A. Roela
Faculdade de Medicina, Universidade de So Paulo, So Paulo, Brazil
3 shared publications- Vinicius Yu Okubo
University of São Paulo
3 shared publications B. S. Shivaram
University of Virginia
2 shared publications- Carlos Shimizu
Instituto do Câncer do Estado de São Paulo, São Paulo, Brazil
2 shared publications
Latest Publications
- Open Access
- 6 September 2026
- Published by Wiley in International Journal of Imaging Systems and Technology
- Vol. 36 (5) , e70436
A Comprehensive Study of Transfer Learning, Resolution Reduction, and Multi‐View Classification for Breast Cancer Detection in Mammograms
Mammography, an X‐ray–based imaging technique, remains central to the early detection of breast cancer. Recent advances in artificial intelligence have enabled increasingly sophisticated computer‐aided diagnostic methods, evolving from patch‐based classifiers to whole‐image approaches and then to multi‐view architectures that jointly analyze complementary projections. Despite this progress, several critical questions remain unanswered. Although individual aspects of CNN‐based mammogram classification—such as patch pretraining, backbone transferability, and multi‐view fusion—have been studied separately in prior work, no unified comparison has examined them under controlled conditions across datasets of contrasting image quality. In this study, we close that gap through a systematic empirical investigation organized around five interrelated questions: (1) the role of patch classifiers, (2) the transferability of natural‐image‐trained backbones, (3) the advantages of “learn‐to‐resize” over conventional downscaling, (4) the contribution of multi‐view integration, and (5) the robustness of findings across varying image quality. Questions (3) and (5) are addressed here for the first time, while questions (1), (2), and (4) are re‐examined within a unified protocol that reveals dataset‐quality dependencies invisible to prior single‐dataset studies. Beyond benchmarking, our experiments demonstrate clear performance gains over prior work. For the CBIS‐DDSM dataset, we improved single‐view AUC from 0.8153 to 0.8343, and multiple‐view AUC from 0.8483 to 0.8658. We also introduce a comparative protocol that places single‐ and multi‐view classifiers on the same evaluation footing—by aggregating per‐view single‐view predictions into per‐exam scores—enabling a statistically valid comparison between the two; using this protocol, we measured a 0.0217 AUC gain from extending single‐ to multi‐view analysis on CBIS‐DDSM. On the complete VinDr‐Mammo dataset, the multiple‐view approach further improved results, achieving a 0.0492 AUC increase over single view and reaching 0.8511 AUC overall. These results establish new state‐of‐the‐art benchmarks, providing clear evidence of the advantages of multi‐view architectures for mammogram interpretation. Beyond performance, our analysis offers principled insights into model design and transfer learning strategies, contributing to the development of more accurate and reliable breast cancer screening tools. The inference code and trained models are publicly available at https://github.com/dpetrini/multiple‐view .
- 6 September 2026
- Published by Wiley in International Journal of Imaging Systems and Technology
- Vol. 36 (5) , e70436
Open AccessA Comprehensive Study of Transfer Learning, Resolution Reduction, and Multi‐View Classification for Breast Cancer Detection in Mammograms
Mammography, an X‐ray–based imaging technique, remains central to the early detection of breast cancer. Recent advances in artificial intelligence have enabled increasingly sophisticated computer‐aided diagnostic methods, evolving from patch‐based classifiers to whole‐image approaches and then to multi‐view architectures that jointly analyze complementary projections. Despite this progress, several critical questions remain unanswered. Although individual aspects of CNN‐based mammogram classification—such as patch pretraining, backbone transferability, and multi‐view fusion—have been studied separately in prior work, no unified comparison has examined them under controlled conditions across datasets of contrasting image quality. In this study, we close that gap through a systematic empirical investigation organized around five interrelated questions: (1) the role of patch classifiers, (2) the transferability of natural‐image‐trained backbones, (3) the advantages of “learn‐to‐resize” over conventional downscaling, (4) the contribution of multi‐view integration, and (5) the robustness of findings across varying image quality. Questions (3) and (5) are addressed here for the first time, while questions (1), (2), and (4) are re‐examined within a unified protocol that reveals dataset‐quality dependencies invisible to prior single‐dataset studies. Beyond benchmarking, our experiments demonstrate clear performance gains over prior work. For the CBIS‐DDSM dataset, we improved single‐view AUC from 0.8153 to 0.8343, and multiple‐view AUC from 0.8483 to 0.8658. We also introduce a comparative protocol that places single‐ and multi‐view classifiers on the same evaluation footing—by aggregating per‐view single‐view predictions into per‐exam scores—enabling a statistically valid comparison between the two; using this protocol, we measured a 0.0217 AUC gain from extending single‐ to multi‐view analysis on CBIS‐DDSM. On the complete VinDr‐Mammo dataset, the multiple‐view approach further improved results, achieving a 0.0492 AUC increase over single view and reaching 0.8511 AUC overall. These results establish new state‐of‐the‐art benchmarks, providing clear evidence of the advantages of multi‐view architectures for mammogram interpretation. Beyond performance, our analysis offers principled insights into model design and transfer learning strategies, contributing to the development of more accurate and reliable breast cancer screening tools. The inference code and trained models are publicly available at https://github.com/dpetrini/multiple‐view .
- Open Access
- 24 August 2026
- Published by American Physical Society (APS) in APS Open Science
- Vol. 1
Machine-learning-assisted characterization of labyrinthine pattern transitions
Labyrinthine structures are ubiquitous in pattern-forming systems arising from a wide variety of physical, chemical, and biological processes. Yet, their characterization remains challenging due to the lack of long-range order and the resultant complicated morphologies. Here, we present an approach for characterizing labyrinthine patterns based on the identification and the correlation analysis of their topological defects. We employ advanced machine-learning-based-pattern-recognition techniques to identify the types and locations of topological defects of the local stripe ordering. Applying this method to single-crystal Bi-substituted yttrium iron garnet films, we uncover a distinct morphological transition between two zero-field labyrinthine structures. Crucially, the pair distribution functions of the topological defects reveal subtle differences between labyrinthine structures, which are not discernible from conventional global analysis. By systematically analyzing the spatial correlations and geometric properties of these defects, we provide insights into the athermal dynamics that govern the observed morphological transitions. Our work demonstrates that machine-learning-based recognition techniques enable systematic studies of rich and complex labyrinthine-type structures universal to many pattern-formation systems.
- 24 August 2026
- Published by American Physical Society (APS) in APS Open Science
- Vol. 1
Open AccessMachine-learning-assisted characterization of labyrinthine pattern transitions
Labyrinthine structures are ubiquitous in pattern-forming systems arising from a wide variety of physical, chemical, and biological processes. Yet, their characterization remains challenging due to the lack of long-range order and the resultant complicated morphologies. Here, we present an approach for characterizing labyrinthine patterns based on the identification and the correlation analysis of their topological defects. We employ advanced machine-learning-based-pattern-recognition techniques to identify the types and locations of topological defects of the local stripe ordering. Applying this method to single-crystal Bi-substituted yttrium iron garnet films, we uncover a distinct morphological transition between two zero-field labyrinthine structures. Crucially, the pair distribution functions of the topological defects reveal subtle differences between labyrinthine structures, which are not discernible from conventional global analysis. By systematically analyzing the spatial correlations and geometric properties of these defects, we provide insights into the athermal dynamics that govern the observed morphological transitions. Our work demonstrates that machine-learning-based recognition techniques enable systematic studies of rich and complex labyrinthine-type structures universal to many pattern-formation systems.
- 16 June 2026
- Published by American Physical Society (APS) in Physical Review E
- Vol. 114 (2) , 024208
Coarsening dynamics of fingerprint labyrinthine patterns: Machine learning assisted characterization
Fingerprint labyrinthine patterns exhibit a level of structural complexity beyond simple stripe phases, combining local stripe order with a dense network of pointlike defects. Unlike symmetry-breaking phases, where coarsening proceeds via diffusive defect annihilation, or conventional stripe phases, where curvature-driven motion of extended grain boundaries dominates, the coarsening of fingerprint labyrinths is governed primarily by localized junction and terminal defects. Using the Turing-Swift-Hohenberg equation, we study the nonequilibrium relaxation of fingerprint labyrinthine patterns following a quench. To go beyond conventional Fourier-based diagnostics, we employ a template-matching convolutional neural network to identify and track junctions and terminals directly in real space, enabling a quantitative characterization of defect statistics and spatial correlations. We show that, although these pointlike defects drive coarsening, their motion is strongly constrained by the surrounding stripe geometry, leading to slow, nondiffusive dynamics that are qualitatively distinct from both conventional phase ordering and stripe coarsening. Together, these results establish defect-mediated dynamics as the central organizing principle of fingerprint labyrinthine coarsening and demonstrate the effectiveness of machine learning-assisted approaches for complex pattern-forming systems.
- 16 June 2026
- Published by American Physical Society (APS) in Physical Review E
- Vol. 114 (2) , 024208
Coarsening dynamics of fingerprint labyrinthine patterns: Machine learning assisted characterization
Fingerprint labyrinthine patterns exhibit a level of structural complexity beyond simple stripe phases, combining local stripe order with a dense network of pointlike defects. Unlike symmetry-breaking phases, where coarsening proceeds via diffusive defect annihilation, or conventional stripe phases, where curvature-driven motion of extended grain boundaries dominates, the coarsening of fingerprint labyrinths is governed primarily by localized junction and terminal defects. Using the Turing-Swift-Hohenberg equation, we study the nonequilibrium relaxation of fingerprint labyrinthine patterns following a quench. To go beyond conventional Fourier-based diagnostics, we employ a template-matching convolutional neural network to identify and track junctions and terminals directly in real space, enabling a quantitative characterization of defect statistics and spatial correlations. We show that, although these pointlike defects drive coarsening, their motion is strongly constrained by the surrounding stripe geometry, leading to slow, nondiffusive dynamics that are qualitatively distinct from both conventional phase ordering and stripe coarsening. Together, these results establish defect-mediated dynamics as the central organizing principle of fingerprint labyrinthine coarsening and demonstrate the effectiveness of machine learning-assisted approaches for complex pattern-forming systems.
- Open Access
- 18 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 30431-30440
How Does Fourier Analysis Network Work? A Mechanism Analysis and a New Dual-Activation Layer Proposal
Fourier Analysis Network (FAN) was recently proposed as a simple way to improve neural network performance by replacing part of ReLU activations with sine and cosine functions. Although several studies have reported small but consistent gains across tasks, the underlying mechanism behind these improvements has remained unclear. In this work, we show that only the sine activation contributes positively to performance, whereas the cosine activation tends to be detrimental. Our analysis reveals that the improvement is not a consequence of the sine function’s periodic nature; instead, it stems from the function’s local behavior near x = 0, where its non-zero derivative mitigates the vanishing-gradient problem. We further show that FAN primarily alleviates the dying-ReLU problem, in which a neuron consistently receives negative inputs, produces zero gradients, and stops learning. Although modern ReLU-like activations, such as Leaky ReLU, GELU, and Swish, reduce ReLU’s zero-gradient region, they still contain input domains where gradients remain significantly diminished, contributing to slower optimization and hindering rapid convergence. FAN addresses this limitation by introducing a more stable gradient pathway. This analysis shifts the understanding of FAN’s benefits from a spectral interpretation to a concrete analysis of training dynamics, leading to the development of the Dual-Activation Layer (DAL), a more efficient convergence accelerator. We evaluate DAL on three tasks: classification of noisy sinusoidal signals versus pure noise, MNIST digit classification, and ECG-based biometric recognition. In all cases, DAL models converge faster and achieve equal or higher validation accuracy compared to models with conventional activations.
- 18 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 30431-30440
Open AccessHow Does Fourier Analysis Network Work? A Mechanism Analysis and a New Dual-Activation Layer Proposal
Fourier Analysis Network (FAN) was recently proposed as a simple way to improve neural network performance by replacing part of ReLU activations with sine and cosine functions. Although several studies have reported small but consistent gains across tasks, the underlying mechanism behind these improvements has remained unclear. In this work, we show that only the sine activation contributes positively to performance, whereas the cosine activation tends to be detrimental. Our analysis reveals that the improvement is not a consequence of the sine function’s periodic nature; instead, it stems from the function’s local behavior near x = 0, where its non-zero derivative mitigates the vanishing-gradient problem. We further show that FAN primarily alleviates the dying-ReLU problem, in which a neuron consistently receives negative inputs, produces zero gradients, and stops learning. Although modern ReLU-like activations, such as Leaky ReLU, GELU, and Swish, reduce ReLU’s zero-gradient region, they still contain input domains where gradients remain significantly diminished, contributing to slower optimization and hindering rapid convergence. FAN addresses this limitation by introducing a more stable gradient pathway. This analysis shifts the understanding of FAN’s benefits from a spectral interpretation to a concrete analysis of training dynamics, leading to the development of the Dual-Activation Layer (DAL), a more efficient convergence accelerator. We evaluate DAL on three tasks: classification of noisy sinusoidal signals versus pure noise, MNIST digit classification, and ECG-based biometric recognition. In all cases, DAL models converge faster and achieve equal or higher validation accuracy compared to models with conventional activations.
- Open Access
- 3 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 92419-92430
Characterization of Magnetic Labyrinthine Structures Through Junctions and Terminals Detection Using Template Matching and CNN
Defects influence diverse properties of materials, shaping their structural, mechanical, and electronic characteristics. Among a variety of materials exhibiting unique defects, magnets exhibit diverse nano- to micro-scale defects and have been intensively studied in materials science. Specifically, defects in magnetic labyrinthine patterns, called junctions and terminals, serve as the canonical targets of the research. While detecting and characterizing such defects is crucial for understanding magnets, systematically investigating large-scale images containing over a thousand closely packed junctions and terminals remains a formidable challenge. This study introduces a new technique called TM-CNN (Template Matching - Convolutional Neural Network) designed to detect a multitude of small objects in images, such as the defects in magnetic labyrinthine patterns. TM-CNNwas used to identify 641,649 such structures in 444 experimental images, and the results were explored to deepen understanding of magnetic materials. It employs a two-stage detection approach combining template matching, used in initial detection, with a convolutional neural network, used to eliminate incorrect identifications. To train a CNN classifier, it is necessary to annotate a large number of training images. This difficulty prevents the use of CNN in many practical applications. TM-CNN significantly reduces the manual workload for creating training images by automatically making most of the annotations and leaving only a small number of corrections to human reviewers. In testing, TM-CNN achieved an impressive F1 score of 0.991, far outperforming traditional template matching and CNN-based object detection algorithms.
- 3 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 92419-92430
Open AccessCharacterization of Magnetic Labyrinthine Structures Through Junctions and Terminals Detection Using Template Matching and CNN
Defects influence diverse properties of materials, shaping their structural, mechanical, and electronic characteristics. Among a variety of materials exhibiting unique defects, magnets exhibit diverse nano- to micro-scale defects and have been intensively studied in materials science. Specifically, defects in magnetic labyrinthine patterns, called junctions and terminals, serve as the canonical targets of the research. While detecting and characterizing such defects is crucial for understanding magnets, systematically investigating large-scale images containing over a thousand closely packed junctions and terminals remains a formidable challenge. This study introduces a new technique called TM-CNN (Template Matching - Convolutional Neural Network) designed to detect a multitude of small objects in images, such as the defects in magnetic labyrinthine patterns. TM-CNNwas used to identify 641,649 such structures in 444 experimental images, and the results were explored to deepen understanding of magnetic materials. It employs a two-stage detection approach combining template matching, used in initial detection, with a convolutional neural network, used to eliminate incorrect identifications. To train a CNN classifier, it is necessary to annotate a large number of training images. This difficulty prevents the use of CNN in many practical applications. TM-CNN significantly reduces the manual workload for creating training images by automatically making most of the annotations and leaving only a small number of corrections to human reviewers. In testing, TM-CNN achieved an impressive F1 score of 0.991, far outperforming traditional template matching and CNN-based object detection algorithms.
- Open Access
- 10 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 9082-9096
Super-Resolution of Color Halftone Images Using Convolutional Neural Networks
In this work, we investigate how to increase the resolution of color halftone images using convolutional neural networks (CNNs). As far as we know, this is the first work that increases resolution of color halftone images using a CNN-based solution. For this task, we first train the well-known Enhanced Deep Super-Resolution (EDSR) network with halftone images to obtain the Halftone-EDSR model. We argue that it is not possible to use conventional data augmentation techniques in this problem, due to the peculiar texture of halftone images. We present cropping as a viable data augmentation technique. Using cropping and image patches as training samples, we substantially speed up the training and get better quality models. We compare the independent channel model (which increases the resolution of each of the CMYK channels independently and then merges them) with the joint channel model (which increases the resolution of all four image channels at once) and conclude that the the latter is superior to the first. We experimentally demonstrate that the proposed Halftone-EDSR is superior to all previous techniques, both for pre-print and post-print halftone images. However, Halftone-EDSR can generate upsampled images with Moiré patterns. To minimize Moiré patterns, we propose a new network model called Halftone-Net. We use the Fast Fourier Transform, followed by a CNN, to detect the strong presence of Moiré patterns in halftone images and demonstrate that Halftone-Net generates fewer images with strong Moiré patterns than Halftone-EDSR.
- 10 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 9082-9096
Open AccessSuper-Resolution of Color Halftone Images Using Convolutional Neural Networks
In this work, we investigate how to increase the resolution of color halftone images using convolutional neural networks (CNNs). As far as we know, this is the first work that increases resolution of color halftone images using a CNN-based solution. For this task, we first train the well-known Enhanced Deep Super-Resolution (EDSR) network with halftone images to obtain the Halftone-EDSR model. We argue that it is not possible to use conventional data augmentation techniques in this problem, due to the peculiar texture of halftone images. We present cropping as a viable data augmentation technique. Using cropping and image patches as training samples, we substantially speed up the training and get better quality models. We compare the independent channel model (which increases the resolution of each of the CMYK channels independently and then merges them) with the joint channel model (which increases the resolution of all four image channels at once) and conclude that the the latter is superior to the first. We experimentally demonstrate that the proposed Halftone-EDSR is superior to all previous techniques, both for pre-print and post-print halftone images. However, Halftone-EDSR can generate upsampled images with Moiré patterns. To minimize Moiré patterns, we propose a new network model called Halftone-Net. We use the Fast Fourier Transform, followed by a CNN, to detect the strong presence of Moiré patterns in halftone images and demonstrate that Halftone-Net generates fewer images with strong Moiré patterns than Halftone-EDSR.
- Open Access
- 25 October 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 112529-112538
Using Convolutional Neural Network to Automate ACR MRI Low-Contrast Detectability Test
According to the American College of Radiology (ACR), the performance of magnetic resonance imaging (MRI) scanners should be monitored using phantom images acquired weekly. These quality assurance images are usually analyzed by a technician, but automated analysis has been proposed to reduce costs and improve repeatability. Reports on the automation of low-contrast detectability tests are scarce, and none can completely replace human work. In previous works, we have demonstrated that machine learning methods can be used to learn the subtleties of image quality and the visual assessment of technicians. We showed that machines are able to mimic human perception quite accurately. In these works, we used hand-designed image quality features. In the present work, we use a deep learning method to automatically design appropriate image features. By training this network on a large base with visual assessments from multiple technicians, we show that the machine can be taught to assess MRI image quality better than any technician alone, justifying its widespread adoption. Our dataset contained 12,000 binary responses to the detectability of low-contrast structures (“holes”). We used the median of the technicians’ responses as the gold standard. To increase statistical power, we repeated training and testing 5 times, using 5-fold cross-validation. We obtained a mean AUC (area under the ROC curve) of 0.983± 0.003. At the point of equal error rate, the mean accuracy, sensitivity and specificity were 93.2±0.7%, numbers higher than those achieved by any technician alone. Applying the obtained model to a completely independent test dataset with 10,800 structures, we obtained an AUC of 0.979. The predictions of our model in classifying spokes (sets of 3 holes) agree in 93.83% of the cases with the median of the responses of the technicians. These results again are better than the responses of any individual technician. We conclude that the ACR test can be performed by a machine with greater reliability than individual technicians.
- 25 October 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 112529-112538
Open AccessUsing Convolutional Neural Network to Automate ACR MRI Low-Contrast Detectability Test
According to the American College of Radiology (ACR), the performance of magnetic resonance imaging (MRI) scanners should be monitored using phantom images acquired weekly. These quality assurance images are usually analyzed by a technician, but automated analysis has been proposed to reduce costs and improve repeatability. Reports on the automation of low-contrast detectability tests are scarce, and none can completely replace human work. In previous works, we have demonstrated that machine learning methods can be used to learn the subtleties of image quality and the visual assessment of technicians. We showed that machines are able to mimic human perception quite accurately. In these works, we used hand-designed image quality features. In the present work, we use a deep learning method to automatically design appropriate image features. By training this network on a large base with visual assessments from multiple technicians, we show that the machine can be taught to assess MRI image quality better than any technician alone, justifying its widespread adoption. Our dataset contained 12,000 binary responses to the detectability of low-contrast structures (“holes”). We used the median of the technicians’ responses as the gold standard. To increase statistical power, we repeated training and testing 5 times, using 5-fold cross-validation. We obtained a mean AUC (area under the ROC curve) of 0.983± 0.003. At the point of equal error rate, the mean accuracy, sensitivity and specificity were 93.2±0.7%, numbers higher than those achieved by any technician alone. Applying the obtained model to a completely independent test dataset with 10,800 structures, we obtained an AUC of 0.979. The predictions of our model in classifying spokes (sets of 3 holes) agree in 93.83% of the cases with the median of the responses of the technicians. These results again are better than the responses of any individual technician. We conclude that the ACR test can be performed by a machine with greater reliability than individual technicians.
- Open Access
- 21 July 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 77723-77731
Breast Cancer Diagnosis in Two-View Mammography Using End-to-End Trained EfficientNet-Based Convolutional Network
Some recent studies have described deep convolutional neural networks to diagnose breast cancer in mammograms with similar or even superior performance to that of human experts. One of the best techniques does two transfer learnings: the first uses a model trained on natural images to create a “patch classifier” that categorizes small subimages; the second uses the patch classifier to scan the whole mammogram and create the “single-view whole-image classifier”. We propose to make a third transfer learning to obtain a “two-view classifier” to use the two mammographic views: bilateral craniocaudal and mediolateral oblique. We use EfficientNet as the basis of our model. We “end-to-end” train the entire system using CBIS-DDSM dataset. To ensure statistical robustness, we test our system twice using: (a) 5-fold cross validation; and (b) the original training/test division of the dataset. Our technique reached an AUC of 0.9344 using 5-fold cross validation (accuracy, sensitivity and specificity are 85.13% at the equal error rate point of ROC). Using the original dataset division, our technique achieved an AUC of 0.8483, as far as we know the highest reported AUC for this problem, although the subtle differences in the testing conditions of each work do not allow for an accurate comparison. The inference code and model are available at https://github.com/dpetrini/two-views-classifier
- 21 July 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 77723-77731
Open AccessBreast Cancer Diagnosis in Two-View Mammography Using End-to-End Trained EfficientNet-Based Convolutional Network
Some recent studies have described deep convolutional neural networks to diagnose breast cancer in mammograms with similar or even superior performance to that of human experts. One of the best techniques does two transfer learnings: the first uses a model trained on natural images to create a “patch classifier” that categorizes small subimages; the second uses the patch classifier to scan the whole mammogram and create the “single-view whole-image classifier”. We propose to make a third transfer learning to obtain a “two-view classifier” to use the two mammographic views: bilateral craniocaudal and mediolateral oblique. We use EfficientNet as the basis of our model. We “end-to-end” train the entire system using CBIS-DDSM dataset. To ensure statistical robustness, we test our system twice using: (a) 5-fold cross validation; and (b) the original training/test division of the dataset. Our technique reached an AUC of 0.9344 using 5-fold cross validation (accuracy, sensitivity and specificity are 85.13% at the equal error rate point of ROC). Using the original dataset division, our technique achieved an AUC of 0.8483, as far as we know the highest reported AUC for this problem, although the subtle differences in the testing conditions of each work do not allow for an accurate comparison. The inference code and model are available at https://github.com/dpetrini/two-views-classifier
- 1 July 2021
- Published by American Association for Cancer Research (AACR)
- Vol. 81 (13 Supplem) , 181
Abstract 181: High-accuracy breast cancer detection in mammography using EfficientNet and end-to-end training
Background:Breast cancer (BC) is the second most common cancer among women. BC screening is usually based on mammography interpreted by radiologists. Recently, some researchers have used deep learning to automatically diagnose BC in mammography and so assist radiologists. The progress of BC detection algorithms can be measured by their performance on public datasets. The CBIS-DDSM is a widely used public dataset composed of scanned mammographies, equally divided into malignant and non-malignant (benign) images. Each image is accompanied by the segmentation of the lesion. Shen et al. (Nature Sci. Rep., 2019) presented a BC detection algorithm using an “end-to-end” approach to train deep neural networks. In this algorithm, a patch classifier is first trained to classify local image patches. The patch classifier's weights are then used to initialize the whole image classifier, that is refined using datasets with the cancer status of the whole image. They achieved an AUC of 0.87 [0.84, 0.90] in classifying CBIS-DDSM images, using their best single-model, single-view breast classifier. They used ResNet (He et al., CVPR 2016) as the basis of their algorithm. Our hypothesis was that replacing the old ResNet with the modern EfficientNet (Tan et al., arXiv 2019) and MobileNetV2 (Sandler et al.,CVPR 2018) would result in greater accuracy. Methods:We tested many different models, to conclude that the best model is obtained using EfficientNet-B4 as the base model, with a MobileNetV2 block at the top, followed by a dense layer with two output categories. We trained the patch classifier using 52,528 patches with 224x224 pixels extracted from CBIS-DDSM. From each image, we extracted 20 patches: 10 patches containing the lesion and 10 from the background (without lesion). The patch classifier weights were then used to initialize the whole image classifier, that was trained using the end-to-end approach with CBIS-DDSM images resized to 1152x896 pixels, with data augmentation. The training was performed using a step learning rate of 1e-4 for the first 20 epochs then 1e-5 for the remaining 10 and batch size of 4, using 10-fold cross-validation. We used 81% of the dataset for training, 9% for validation and 10% for testing. Results:We obtained an AUC of 0.8963±0.06, using a single-model, single-view classifier and without test-time data augmentation. Conclusions:Using EfficientNet and MobileNetV2 as the basis of the BC detection algorithm (instead of ResNet), we obtained an improvement in classifying CBIS-DDSM images into malignant/non-malignant: AUC has increased from 0.87 to 0.896. Our AUC is also larger than other recent papers in the literature, such as Shu et al. (IEEE Trans Med. Image, 2020) that achieved an AUC of 0.838 in the same CBIS-DDSM dataset. Citation Format: Daniel G. Petrini, Carlos Shimizu, Gabriel V. Valente, Guilherme Folgueira, Guilherme A. Novaes, Maria L. Katayama, Pedro Serio, Rosimeire A. Roela, Tatiana C. Tucunduva, Maria Aparecida A. Folgueira, Hae Y. Kim. High-accuracy breast cancer detection in mammography using EfficientNet and end-to-end training [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 181.
- 1 July 2021
- Published by American Association for Cancer Research (AACR)
- Vol. 81 (13 Supplem) , 181
Abstract 181: High-accuracy breast cancer detection in mammography using EfficientNet and end-to-end training
Background:Breast cancer (BC) is the second most common cancer among women. BC screening is usually based on mammography interpreted by radiologists. Recently, some researchers have used deep learning to automatically diagnose BC in mammography and so assist radiologists. The progress of BC detection algorithms can be measured by their performance on public datasets. The CBIS-DDSM is a widely used public dataset composed of scanned mammographies, equally divided into malignant and non-malignant (benign) images. Each image is accompanied by the segmentation of the lesion. Shen et al. (Nature Sci. Rep., 2019) presented a BC detection algorithm using an “end-to-end” approach to train deep neural networks. In this algorithm, a patch classifier is first trained to classify local image patches. The patch classifier's weights are then used to initialize the whole image classifier, that is refined using datasets with the cancer status of the whole image. They achieved an AUC of 0.87 [0.84, 0.90] in classifying CBIS-DDSM images, using their best single-model, single-view breast classifier. They used ResNet (He et al., CVPR 2016) as the basis of their algorithm. Our hypothesis was that replacing the old ResNet with the modern EfficientNet (Tan et al., arXiv 2019) and MobileNetV2 (Sandler et al.,CVPR 2018) would result in greater accuracy. Methods:We tested many different models, to conclude that the best model is obtained using EfficientNet-B4 as the base model, with a MobileNetV2 block at the top, followed by a dense layer with two output categories. We trained the patch classifier using 52,528 patches with 224x224 pixels extracted from CBIS-DDSM. From each image, we extracted 20 patches: 10 patches containing the lesion and 10 from the background (without lesion). The patch classifier weights were then used to initialize the whole image classifier, that was trained using the end-to-end approach with CBIS-DDSM images resized to 1152x896 pixels, with data augmentation. The training was performed using a step learning rate of 1e-4 for the first 20 epochs then 1e-5 for the remaining 10 and batch size of 4, using 10-fold cross-validation. We used 81% of the dataset for training, 9% for validation and 10% for testing. Results:We obtained an AUC of 0.8963±0.06, using a single-model, single-view classifier and without test-time data augmentation. Conclusions:Using EfficientNet and MobileNetV2 as the basis of the BC detection algorithm (instead of ResNet), we obtained an improvement in classifying CBIS-DDSM images into malignant/non-malignant: AUC has increased from 0.87 to 0.896. Our AUC is also larger than other recent papers in the literature, such as Shu et al. (IEEE Trans Med. Image, 2020) that achieved an AUC of 0.838 in the same CBIS-DDSM dataset. Citation Format: Daniel G. Petrini, Carlos Shimizu, Gabriel V. Valente, Guilherme Folgueira, Guilherme A. Novaes, Maria L. Katayama, Pedro Serio, Rosimeire A. Roela, Tatiana C. Tucunduva, Maria Aparecida A. Folgueira, Hae Y. Kim. High-accuracy breast cancer detection in mammography using EfficientNet and end-to-end training [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 181.
- 1 July 2021
- Published by American Association for Cancer Research (AACR) in Cancer Research
- Vol. 81 (13_Supplem) , 183
Abstract 183: End-to-end training of convolutional network for breast cancer detection in two-view mammography
Background:Early computer-aided detection systems for mammography have failed to improve the performance of radiologists. With the remarkable success of deep learning, some recent studies have described computer systems with similar or even superior performance to that of human experts. Among them, Shen et al. (Nature Sci. Rep., 2019) present a promising “end-to-end” training approach. Instead of training a convolutional net with whole mammograms, they first train a “patch classifier” that recognizes lesions in small subimages. Then, they generalize the patch classifier to “whole image classifier” using the property of fully convolutional networks and the end-to-end approach. Using this strategy, the authors have obtained a per-image AUC of 0.87 [0.84, 0.90] in the CBIS-DDSM dataset. Standard mammography consists of two views for each breast: bilateral craniocaudal (CC) and mediolateral oblique (MLO). The algorithm proposed by Shen et al. processes only single-view mammography. We extend their work, presenting the end-to-end training of convolutional net for two-view mammography. Methods:First, we reproduced Shen et al.'s work, using the CBIS-DDSM dataset. We trained a ResNet50-based net for classifying patches with 224x224 pixels using segmented lesions. Then, the weights of the patch classifier were transferred to the whole image single-view classifier, obtained by removing the dense layers from the patch classifier and stacking one ResNet block at the top. This single-view classifier was trained using full images from the same dataset. Trying to replicate Shen et al.'s work, we obtained an AUC of 0.8524±0.0560, less than 0.87 reported in the original paper. We attribute this worsening to the fact that we are using only 2260 images with two views, instead of 2478 images from the original work. Finally, we built the two-view classifier that receives CC and MLO views as input. This classifier has inside two copies of the patch classifier, loaded with the weights from the single-view classifier. The features extracted by the two patch classifiers are concatenated and submitted to the ResNet block. The two-view classifier is end-to-end trained using full images, refining all its weights, including those inside the two patch classifiers. Results:The two-view classifier yielded an AUC of 0.9199±0.0623 in 5-fold cross-validation to classify mammographies into malignant/non-malignant, using single-model and without test-time data augmentation. This is better than the Shen et al.'s AUC (0.87), our single-view AUC (0.85). Zhang et al. (Plos One, 2020) present another two-view algorithm (without end-to-end training) with AUC of 0.95. However, this work cannot directly be compared with ours, as it was tested on a different set of images. Conclusions:We presented end-to-end training of convolutional net for two-view mammography. Our system's AUC was 0.92, better than the 0.87 obtained by the previous single-view system. Citation Format: Daniel G. Petrini, Carlos Shimizu, Gabriel V. Valente, Guilherme Folgueira, Guilherme A. Novaes, Maria L. Katayama, Pedro Serio, Rosimeire A. Roela, Tatiana C. Tucunduva, Maria Aparecida A. Folgueira, Hae Y. Kim. End-to-end training of convolutional network for breast cancer detection in two-view mammography [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 183.
- 1 July 2021
- Published by American Association for Cancer Research (AACR) in Cancer Research
- Vol. 81 (13_Supplem) , 183
Abstract 183: End-to-end training of convolutional network for breast cancer detection in two-view mammography
Background:Early computer-aided detection systems for mammography have failed to improve the performance of radiologists. With the remarkable success of deep learning, some recent studies have described computer systems with similar or even superior performance to that of human experts. Among them, Shen et al. (Nature Sci. Rep., 2019) present a promising “end-to-end” training approach. Instead of training a convolutional net with whole mammograms, they first train a “patch classifier” that recognizes lesions in small subimages. Then, they generalize the patch classifier to “whole image classifier” using the property of fully convolutional networks and the end-to-end approach. Using this strategy, the authors have obtained a per-image AUC of 0.87 [0.84, 0.90] in the CBIS-DDSM dataset. Standard mammography consists of two views for each breast: bilateral craniocaudal (CC) and mediolateral oblique (MLO). The algorithm proposed by Shen et al. processes only single-view mammography. We extend their work, presenting the end-to-end training of convolutional net for two-view mammography. Methods:First, we reproduced Shen et al.'s work, using the CBIS-DDSM dataset. We trained a ResNet50-based net for classifying patches with 224x224 pixels using segmented lesions. Then, the weights of the patch classifier were transferred to the whole image single-view classifier, obtained by removing the dense layers from the patch classifier and stacking one ResNet block at the top. This single-view classifier was trained using full images from the same dataset. Trying to replicate Shen et al.'s work, we obtained an AUC of 0.8524±0.0560, less than 0.87 reported in the original paper. We attribute this worsening to the fact that we are using only 2260 images with two views, instead of 2478 images from the original work. Finally, we built the two-view classifier that receives CC and MLO views as input. This classifier has inside two copies of the patch classifier, loaded with the weights from the single-view classifier. The features extracted by the two patch classifiers are concatenated and submitted to the ResNet block. The two-view classifier is end-to-end trained using full images, refining all its weights, including those inside the two patch classifiers. Results:The two-view classifier yielded an AUC of 0.9199±0.0623 in 5-fold cross-validation to classify mammographies into malignant/non-malignant, using single-model and without test-time data augmentation. This is better than the Shen et al.'s AUC (0.87), our single-view AUC (0.85). Zhang et al. (Plos One, 2020) present another two-view algorithm (without end-to-end training) with AUC of 0.95. However, this work cannot directly be compared with ours, as it was tested on a different set of images. Conclusions:We presented end-to-end training of convolutional net for two-view mammography. Our system's AUC was 0.92, better than the 0.87 obtained by the previous single-view system. Citation Format: Daniel G. Petrini, Carlos Shimizu, Gabriel V. Valente, Guilherme Folgueira, Guilherme A. Novaes, Maria L. Katayama, Pedro Serio, Rosimeire A. Roela, Tatiana C. Tucunduva, Maria Aparecida A. Folgueira, Hae Y. Kim. End-to-end training of convolutional network for breast cancer detection in two-view mammography [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 183.