Student-Pilot Error Prediction via Multimodal Physiological Signals and Tree-Based Models
- 30 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. 18 (1) , 214-227
Abstract
The United States Air Force (USAF) faces significant challenges in training new pilots to meet the growing demand for skilled airmen in various essential operations. While varied in nature, these challenges are underpinned by significant resource constraints, e.g., limited trainer aircraft ability, shortage of instructor pilots, and weather-related limitations. Recent USAF initiatives leverage simulator flights in training programs to ameliorate such issues; however, given that the USAF also faces a lack of available simulator instructors, resource limitations are not entirely circumvented. Therefore, this study aims to alleviate the strain on instructors by building a machine learning pipeline to predict flight errors in simulator flights. By accurately quantifying errors, the burden of the time-consuming duties of instructors can be reduced, allowing them to focus instead on personalized training for student pilots. Leveraging physiological and aircraft data recorded during simulator flights involving aircraft landings, this research uses functional data analysis, modern feature-selection techniques, and efficient tree-based machine learning methods to predict student performance and highlight key predictive physiological modalities. The proposed pipeline, coined the Multimodal Functional Learning for Error Regression (mFLYER), produces various models that demonstrate notable improvements over prior approaches, closing the gap between successful manual and automated grades. Finally, to further examine the applicability of such models, this approach leverages uncertainty quantification through calibrated models, then ensembles, to identify their ability to confidently differentiate between low- and high-error landings, which many models struggle to accomplish. Finally, additional analysis is performed to determine how future research may improve model performance and confidence.Keywords
Funding Information
- Air Force Office of Scientific Research through the Dynamic Data and Information Processing portfolio (23AFCOR003)
- CogPilot Dataset provided by the USAF pursuant to Cooperative (FA8750-19-2-1000)
This publication has 48 references indexed in Scilit:
- Artificial Intelligence in Pilot Training and Education – Towards a Machine Learning Aided Instructor Assistant for Flight Simulators2021
- Machine Learning and Physiological Metrics Enhance Performance AssessmentCommunications in Computer and Information Science, 2019
- Nature inspired feature selection meta-heuristicsArtificial Intelligence Review, 2015
- A survey of functional principal component analysisAStA Advances in Statistical Analysis, 2013
- Transductive conformal predictors2013
- Flow and Its Affective, Cognitive, and Performance-Related Consequences2012
- Filter Methods for Feature Selection – A Comparative Study2007
- Variable Selection Using Random Forests2006
- Algorithmic Learning in a Random World2005
- Inductive Confidence Machines for Regression2002