Chancellor Johnstone
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Chancellor Johnstone has a total of 21 co-authors. Here are the top co-authors by number of shared publications:
- Phillip R. Jenkins
Department of Operational Sciences, Air Force Institute of Technology, Wright-Patterson Air Force Base, OH, USA
3 shared publications - William N. Caballero
Department of Operational Sciences, Air Force Institute of Technology, Wright-Patterson Air Force Base, OH, USA
3 shared publications - Nathan Gaw
Department of Operational Sciences, Air Force Institute of Technology, Wright-Patterson Air Force Base, OH, USA
2 shared publications - Aihua Wood
Air Force Inst Technol, Dept Math & Stat, Wright Patterson AFB, OH 45433 USA
1 shared publication Alex Contarino
1 shared publication- Brandon Harvill
United States Air Force
1 shared publication
Latest Publications
- 30 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. 18 (1) , 214-227
Student-Pilot Error Prediction via Multimodal Physiological Signals and Tree-Based Models
The United States Air Force (USAF) faces significant challenges in training new pilots to meet the growing demand for skilled airmen in various essential operations. While varied in nature, these challenges are underpinned by significant resource constraints, e.g., limited trainer aircraft ability, shortage of instructor pilots, and weather-related limitations. Recent USAF initiatives leverage simulator flights in training programs to ameliorate such issues; however, given that the USAF also faces a lack of available simulator instructors, resource limitations are not entirely circumvented. Therefore, this study aims to alleviate the strain on instructors by building a machine learning pipeline to predict flight errors in simulator flights. By accurately quantifying errors, the burden of the time-consuming duties of instructors can be reduced, allowing them to focus instead on personalized training for student pilots. Leveraging physiological and aircraft data recorded during simulator flights involving aircraft landings, this research uses functional data analysis, modern feature-selection techniques, and efficient tree-based machine learning methods to predict student performance and highlight key predictive physiological modalities. The proposed pipeline, coined the Multimodal Functional Learning for Error Regression (mFLYER), produces various models that demonstrate notable improvements over prior approaches, closing the gap between successful manual and automated grades. Finally, to further examine the applicability of such models, this approach leverages uncertainty quantification through calibrated models, then ensembles, to identify their ability to confidently differentiate between low- and high-error landings, which many models struggle to accomplish. Finally, additional analysis is performed to determine how future research may improve model performance and confidence.
- 30 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. 18 (1) , 214-227
Student-Pilot Error Prediction via Multimodal Physiological Signals and Tree-Based Models
The United States Air Force (USAF) faces significant challenges in training new pilots to meet the growing demand for skilled airmen in various essential operations. While varied in nature, these challenges are underpinned by significant resource constraints, e.g., limited trainer aircraft ability, shortage of instructor pilots, and weather-related limitations. Recent USAF initiatives leverage simulator flights in training programs to ameliorate such issues; however, given that the USAF also faces a lack of available simulator instructors, resource limitations are not entirely circumvented. Therefore, this study aims to alleviate the strain on instructors by building a machine learning pipeline to predict flight errors in simulator flights. By accurately quantifying errors, the burden of the time-consuming duties of instructors can be reduced, allowing them to focus instead on personalized training for student pilots. Leveraging physiological and aircraft data recorded during simulator flights involving aircraft landings, this research uses functional data analysis, modern feature-selection techniques, and efficient tree-based machine learning methods to predict student performance and highlight key predictive physiological modalities. The proposed pipeline, coined the Multimodal Functional Learning for Error Regression (mFLYER), produces various models that demonstrate notable improvements over prior approaches, closing the gap between successful manual and automated grades. Finally, to further examine the applicability of such models, this approach leverages uncertainty quantification through calibrated models, then ensembles, to identify their ability to confidently differentiate between low- and high-error landings, which many models struggle to accomplish. Finally, additional analysis is performed to determine how future research may improve model performance and confidence.
- 26 February 2024
- Published by Springer Nature
- p. 315-344
A Survey of Advances in Multimodal Federated Learning with Applications
- 26 February 2024
- Published by Springer Nature
- p. 315-344
A Survey of Advances in Multimodal Federated Learning with Applications
- 21 February 2024
- Published by Elsevier in Decision Support Systems
- Vol. 180
Responsible machine learning for United States Air Force pilot candidate selection
- 21 February 2024
- Published by Elsevier in Decision Support Systems
- Vol. 180
Responsible machine learning for United States Air Force pilot candidate selection
- 21 December 2023
- Published by Taylor & Francis in The American Statistician
- Vol. 78 (3) , 304-317
Using Conformal Win Probability to Predict the Winners of the Canceled 2020 NCAA Basketball Tournaments
- 21 December 2023
- Published by Taylor & Francis in The American Statistician
- Vol. 78 (3) , 304-317
Using Conformal Win Probability to Predict the Winners of the Canceled 2020 NCAA Basketball Tournaments
- Open Access
- 7 August 2023
- Published by MDPI AG in Mathematics
- Vol. 11 (15) , 3428
Anomaly Detection in the Molecular Structure of Gallium Arsenide Using Convolutional Neural Networks
This paper concerns the development of a machine learning tool to detect anomalies in the molecular structure of Gallium Arsenide. We employ a combination of a CNN and a PCA reconstruction to create the model, using real images taken with an electron microscope in training and testing. The methodology developed allows for the creation of a defect detection model, without any labeled images of defects being required for training. The model performed well on all tests under the established assumptions, allowing for reliable anomaly detection. To the best of our knowledge, such methods are not currently available in the open literature; thus, this work fills a gap in current capabilities.
- 7 August 2023
- Published by MDPI AG in Mathematics
- Vol. 11 (15) , 3428
Open AccessAnomaly Detection in the Molecular Structure of Gallium Arsenide Using Convolutional Neural Networks
This paper concerns the development of a machine learning tool to detect anomalies in the molecular structure of Gallium Arsenide. We employ a combination of a CNN and a PCA reconstruction to create the model, using real images taken with an electron microscope in training and testing. The methodology developed allows for the creation of a defect detection model, without any labeled images of defects being required for training. The model performed well on all tests under the established assumptions, allowing for reliable anomaly detection. To the best of our knowledge, such methods are not currently available in the open literature; thus, this work fills a gap in current capabilities.
- Open Access
- 8 June 2023
- Published by Elsevier in Expert Systems with Applications
- Vol. 231
Toward automated instructor pilots in legacy Air Force systems: Physiology-based flight difficulty classification via machine learning
- 8 June 2023
- Published by Elsevier in Expert Systems with Applications
- Vol. 231
Open AccessToward automated instructor pilots in legacy Air Force systems: Physiology-based flight difficulty classification via machine learning
- Open Access
Constructing Prediction Intervals with Neural Networks: An Empirical Evaluation of Bootstrapping and Conformal Inference Methods
Artificial neural networks (ANNs) are popular tools for accomplishing many machine learning tasks, including predicting continuous outcomes. However, the general lack of confidence measures provided with ANN predictions limit their applicability. Supplementing point predictions with prediction intervals (PIs) is common for other learning algorithms, but the complex structure and training of ANNs renders constructing PIs difficult. This work provides the network design choices and inferential methods for creating better performing PIs with ANNs. A two-step experiment is executed across 11 data sets, including an imaged-based data set. Two distribution-free methods for constructing PIs, bootstrapping and conformal inference, are considered. The results of the first experimental step reveal that the choices inherent to building an ANN affect PI performance. Guidance is provided for optimizing PI performance with respect to each network feature and PI method. In the second step, 20 algorithms for constructing PIs, each using the principles of bootstrapping or conformal inference, are implemented to determine which provides the best performance while maintaining reasonable computational burden. In general, this trade-off is optimized when implementing the cross-conformal method, which maintained interval coverage and efficiency with decreased computational burden.
Open AccessConstructing Prediction Intervals with Neural Networks: An Empirical Evaluation of Bootstrapping and Conformal Inference Methods
Artificial neural networks (ANNs) are popular tools for accomplishing many machine learning tasks, including predicting continuous outcomes. However, the general lack of confidence measures provided with ANN predictions limit their applicability. Supplementing point predictions with prediction intervals (PIs) is common for other learning algorithms, but the complex structure and training of ANNs renders constructing PIs difficult. This work provides the network design choices and inferential methods for creating better performing PIs with ANNs. A two-step experiment is executed across 11 data sets, including an imaged-based data set. Two distribution-free methods for constructing PIs, bootstrapping and conformal inference, are considered. The results of the first experimental step reveal that the choices inherent to building an ANN affect PI performance. Guidance is provided for optimizing PI performance with respect to each network feature and PI method. In the second step, 20 algorithms for constructing PIs, each using the principles of bootstrapping or conformal inference, are implemented to determine which provides the best performance while maintaining reasonable computational burden. In general, this trade-off is optimized when implementing the cross-conformal method, which maintained interval coverage and efficiency with decreased computational burden.
- Open Access
Conformal Uncertainty Sets for Robust Optimization
Decision-making under uncertainty is hugely important for any decisions sensitive to perturbations in observed data. One method of incorporating uncertainty into making optimal decisions is through robust optimization, which minimizes the worst-case scenario over some uncertainty set. We connect conformal prediction regions to robust optimization, providing finite sample valid and conservative ellipsoidal uncertainty sets, aptly named conformal uncertainty sets. In pursuit of this connection we explicitly define Mahalanobis distance as a potential conformity score in full conformal prediction. We also compare the coverage and optimization performance of conformal uncertainty sets, specifically generated with Mahalanobis distance, to traditional ellipsoidal uncertainty sets on a collection of simulated robust optimization examples.
Open AccessConformal Uncertainty Sets for Robust Optimization
Decision-making under uncertainty is hugely important for any decisions sensitive to perturbations in observed data. One method of incorporating uncertainty into making optimal decisions is through robust optimization, which minimizes the worst-case scenario over some uncertainty set. We connect conformal prediction regions to robust optimization, providing finite sample valid and conservative ellipsoidal uncertainty sets, aptly named conformal uncertainty sets. In pursuit of this connection we explicitly define Mahalanobis distance as a potential conformity score in full conformal prediction. We also compare the coverage and optimization performance of conformal uncertainty sets, specifically generated with Mahalanobis distance, to traditional ellipsoidal uncertainty sets on a collection of simulated robust optimization examples.