Loading...

Online Flight Crew Alertness and Fatigue Level Monitoring System Using Machine Learning

Ghaderi, Alireza | 2025

0 Viewed
  1. Type of Document: Ph.D. Dissertation
  2. Language: Farsi
  3. Document No: 58312 (45)
  4. University: Sharif University of Technology
  5. Department: Aerospace Engineering
  6. Advisor(s): Saghafi, Fariborz
  7. Abstract:
  8. Pilot fatigue and increased cognitive load are primary contributors to fatal aviation accidents, especially in general aviation. However, pilots lack real-time, objective systems for monitoring their cognitive states. This dissertation introduces a cost-effective multimodal framework combining gaze tracking, control stick inputs, and Continuous Performance Test (CPT) scores to compute a cognitive performance stability index based on exceedance autocorrelation. Experiments were conducted in a single-engine Cessna simulator under IFR conditions with pilots of varying experience. Simulator data, gaze coordinates, control inputs, and CPT outputs were captured through automated software infrastructure. Following data processing, Continuous Wavelet Transform (CWT) scalograms, eye-hand cross-wavelet correlations, and test indices were extracted. Three deep learning approaches were evaluated, progressing from single-modality convolutional networks with scalogram inputs to multimodal networks integrating scalograms and CPT data, culminating in hybrid models incorporating LSTM layers for temporal pattern analysis. Data scarcity was addressed through transfer learning methods. The multimodal Inception-ResNet network achieved optimal performance while the lighter GoogLeNet variant demonstrated excellent efficiency for real-time embedded implementation. Results show that low-cost sensors combined with advanced signal processing enable accurate real-time monitoring of pilot cognitive stability. The modular software suite provides complete workflow from capture to reporting, ready for integration into Fatigue Risk Management Systems and Safety Management Systems. This framework is adaptable from single-engine simulators to commercial aircraft cockpits
  9. Keywords:
  10. Aviation ; Human Performance ; Human Factors ; Transfer Learning ; Wavelet Transform ; Pilot Fatigue Monitoring ; Multi-Modal Deep Neural Networks ; Cognitive Workload ; Alertness

 Digital Object List

 Bookmark

...see more