Justin M. Johnson
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Justin M. Johnson has a total of 3 co-authors. Here are the top co-authors by number of shared publications:
Taghi M. Khoshgoftaar
College of Engineering and Computer Science, Florida Atlantic University, Boca Raton, Florida 33431, United States
5 shared publications- Taghi M. Khoshgoftaar
Department of Electrical Engineering and Computer Science, Florida Atlantic University, Boca Raton, USA
4 shared publications Robert K. L. Kennedy
College of Engineering and Computer Science, Florida Atlantic University, Boca Raton, Florida 33431, United States
1 shared publication
Latest Publications
- 16 August 2023
- Published by World Scientific Pub Co Pte Ltd in International Journal on Artificial Intelligence Tools
- Vol. 32 (05)
Learning from Highly Imbalanced Big Data with Label Noise
This study explores the effects of class label noise on detecting fraud within three highly imbalanced healthcare fraud data sets containing millions of claims and minority class sizes as small as 0.1%. For each data set, 29 noise distributions are simulated by varying the level of class noise and the distribution of noise between the fraudulent and non-fraudulent classes. Four popular machine learning algorithms are evaluated on each noise distribution using six rounds of five-fold cross-validation. Performance is measured using the area under the precision-recall curve (AUPRC), true positive rate (TPR), and true negative rate (TNR) in order to understand the effect of the noise level, noise distribution, and their interactions. AUPRC results show that negative class noise, i.e. fraudulent samples incorrectly labeled as non-fraudulent, is the most detrimental to model performance. TPR and TNR results show that there are significant trade-offs in class-wise performance as noise transitions between the positive and the negative class. Finally, results reveal how overfitting negatively impacts the classification performance of some learners, and how simple regularization can be used to combat this overfitting and improve classification performance across all noise distributions.
- 16 August 2023
- Published by World Scientific Pub Co Pte Ltd in International Journal on Artificial Intelligence Tools
- Vol. 32 (05)
Learning from Highly Imbalanced Big Data with Label Noise
This study explores the effects of class label noise on detecting fraud within three highly imbalanced healthcare fraud data sets containing millions of claims and minority class sizes as small as 0.1%. For each data set, 29 noise distributions are simulated by varying the level of class noise and the distribution of noise between the fraudulent and non-fraudulent classes. Four popular machine learning algorithms are evaluated on each noise distribution using six rounds of five-fold cross-validation. Performance is measured using the area under the precision-recall curve (AUPRC), true positive rate (TPR), and true negative rate (TNR) in order to understand the effect of the noise level, noise distribution, and their interactions. AUPRC results show that negative class noise, i.e. fraudulent samples incorrectly labeled as non-fraudulent, is the most detrimental to model performance. TPR and TNR results show that there are significant trade-offs in class-wise performance as noise transitions between the positive and the negative class. Finally, results reveal how overfitting negatively impacts the classification performance of some learners, and how simple regularization can be used to combat this overfitting and improve classification performance across all noise distributions.
- 11 May 2023
- Published by Springer Nature in SN Computer Science
- Vol. 4 (4) , 1-14
Data-Centric AI for Healthcare Fraud Detection
Automated methods for detecting fraudulent healthcare providers have the potential to save billions of dollars in healthcare costs and improve the overall quality of patient care. This study presents a data-centric approach to improve healthcare fraud classification performance and reliability using Medicare claims data. Publicly available data from the Centers for Medicare & Medicaid Services (CMS) are used to construct nine large-scale labeled data sets for supervised learning. First, we leverage CMS data to curate the 2013–2019 Part B, Part D, and Durable Medical Equipment, Prosthetics, Orthotics, and Supplies (DMEPOS) Medicare fraud classification data sets. We provide a review of each data set and data preparation techniques to create Medicare data sets for supervised learning and we propose an improved data labeling process. Next, we enrich the original Medicare fraud data sets with up to 58 new provider summary features. Finally, we address a common model evaluation pitfall and propose an adjusted cross-validation technique that mitigates target leakage to provide reliable evaluation results. Each data set is evaluated on the Medicare fraud classification task using extreme gradient boosting and random forest learners, multiple complementary performance metrics, and 95% confidence intervals. Results show that the new enriched data sets consistently outperform the original Medicare data sets that are currently used in related works. Our results encourage the data-centric machine learning workflow and provide a strong foundation for data understanding and preparation techniques for machine learning applications in healthcare fraud.
- 11 May 2023
- Published by Springer Nature in SN Computer Science
- Vol. 4 (4) , 1-14
Data-Centric AI for Healthcare Fraud Detection
Automated methods for detecting fraudulent healthcare providers have the potential to save billions of dollars in healthcare costs and improve the overall quality of patient care. This study presents a data-centric approach to improve healthcare fraud classification performance and reliability using Medicare claims data. Publicly available data from the Centers for Medicare & Medicaid Services (CMS) are used to construct nine large-scale labeled data sets for supervised learning. First, we leverage CMS data to curate the 2013–2019 Part B, Part D, and Durable Medical Equipment, Prosthetics, Orthotics, and Supplies (DMEPOS) Medicare fraud classification data sets. We provide a review of each data set and data preparation techniques to create Medicare data sets for supervised learning and we propose an improved data labeling process. Next, we enrich the original Medicare fraud data sets with up to 58 new provider summary features. Finally, we address a common model evaluation pitfall and propose an adjusted cross-validation technique that mitigates target leakage to provide reliable evaluation results. Each data set is evaluated on the Medicare fraud classification task using extreme gradient boosting and random forest learners, multiple complementary performance metrics, and 95% confidence intervals. Results show that the new enriched data sets consistently outperform the original Medicare data sets that are currently used in related works. Our results encourage the data-centric machine learning workflow and provide a strong foundation for data understanding and preparation techniques for machine learning applications in healthcare fraud.
- Open Access
- 23 November 2022
- Published by Association for Computing Machinery (ACM) in Journal of Data and Information Quality
- Vol. 14 (4) , 1-43
A Survey on Classifying Big Data with Label Noise
Class label noise is a critical component of data quality that directly inhibits the predictive performance of machine learning algorithms. While many data-level and algorithm-level methods exist for treating label noise, the challenges associated with big data call for new and improved methods. This survey addresses these concerns by providing an extensive literature review on treating label noise within big data. We begin with an introduction to the class label noise problem and traditional methods for treating label noise. Next, we present 30 methods for treating class label noise in a range of big data contexts, i.e., high-volume, high-variety, and high-velocity problems. The surveyed works include distributed solutions capable of operating on datasets of arbitrary sizes, deep learning techniques for large-scale datasets with limited clean labels, and streaming techniques for detecting class noise in the presence of concept drift. Common trends and best practices are identified in each of these areas, implementation details are reviewed, empirical results are compared across studies when applicable, and references to 17 open-source projects and programming packages are provided. An emphasis on label noise challenges, solutions, and empirical results as they relate to big data distinguishes this work as a unique contribution that will inspire future research and guide machine learning practitioners.
- 23 November 2022
- Published by Association for Computing Machinery (ACM) in Journal of Data and Information Quality
- Vol. 14 (4) , 1-43
Open AccessA Survey on Classifying Big Data with Label Noise
Class label noise is a critical component of data quality that directly inhibits the predictive performance of machine learning algorithms. While many data-level and algorithm-level methods exist for treating label noise, the challenges associated with big data call for new and improved methods. This survey addresses these concerns by providing an extensive literature review on treating label noise within big data. We begin with an introduction to the class label noise problem and traditional methods for treating label noise. Next, we present 30 methods for treating class label noise in a range of big data contexts, i.e., high-volume, high-variety, and high-velocity problems. The surveyed works include distributed solutions capable of operating on datasets of arbitrary sizes, deep learning techniques for large-scale datasets with limited clean labels, and streaming techniques for detecting class noise in the presence of concept drift. Common trends and best practices are identified in each of these areas, implementation details are reviewed, empirical results are compared across studies when applicable, and references to 17 open-source projects and programming packages are provided. An emphasis on label noise challenges, solutions, and empirical results as they relate to big data distinguishes this work as a unique contribution that will inspire future research and guide machine learning practitioners.
- 5 July 2022
- Published by Springer Nature in SN Computer Science
- Vol. 3 (5) , 1-15
Encoding High-Dimensional Procedure Codes for Healthcare Fraud Detection
- 5 July 2022
- Published by Springer Nature in SN Computer Science
- Vol. 3 (5) , 1-15
Encoding High-Dimensional Procedure Codes for Healthcare Fraud Detection
- 15 May 2021
- Published by Springer Nature in SN Computer Science
- Vol. 2 (4) , 1-15
Medical Provider Embeddings for Healthcare Fraud Detection
- 15 May 2021
- Published by Springer Nature in SN Computer Science
- Vol. 2 (4) , 1-15
Medical Provider Embeddings for Healthcare Fraud Detection
- 25 September 2020
- Published by Springer Nature
- p. 199-227
Thresholding Strategies for Deep Learning with Highly Imbalanced Big Data
- 25 September 2020
- Published by Springer Nature
- p. 199-227
Thresholding Strategies for Deep Learning with Highly Imbalanced Big Data
- 21 June 2020
- Published by Springer Nature in Information Systems Frontiers
- Vol. 22 (5) , 1113-1131
The Effects of Data Sampling with Deep Learning and Highly Imbalanced Big Data
- 21 June 2020
- Published by Springer Nature in Information Systems Frontiers
- Vol. 22 (5) , 1113-1131
The Effects of Data Sampling with Deep Learning and Highly Imbalanced Big Data
- Open Access
- 18 July 2019
- Published by Springer Nature in Journal of Big Data
- Vol. 6 (1) , 1-35
Medicare fraud detection using neural networks
- 18 July 2019
- Published by Springer Nature in Journal of Big Data
- Vol. 6 (1) , 1-35
Open AccessMedicare fraud detection using neural networks
- Top Cited PapersOpen Access
- 19 March 2019
- Published by Springer Nature in Journal of Big Data
- Vol. 6 (1) , 27
Survey on deep learning with class imbalance
- 19 March 2019
- Published by Springer Nature in Journal of Big Data
- Vol. 6 (1) , 27
Top Cited PapersOpen AccessSurvey on deep learning with class imbalance