Late Fusion-Based Multimodal Machine Learning for Driver Fatigue Detection in Emergency Response Driving Scenarios

DOI

10.2991/978-94-6239-705-7_8How to use a DOI?

Keywords

Missing modalities; Ensemble Learning; Emergency Response Driving; Drowsiness Detection; Stacking

Abstract

Globally, road accidents are among the leading causes of death and injury, with driver fatigue being one of the main contributors. Driver fatigue is especially critical in emergency response driving, where factors such as long and overnight shifts are common. This study presents a multimodal driver fatigue detection Machine Learning (ML) model that utilizes a late fusion approach. A range of behavioral and physiological data was used to train base learners with various ML models, and their prediction probabilities were used to train four decision tree-based metaclassifiers (RF, XGB, LGB, and HGB) within a stacking late fusion framework. For a broader exploration, ensemble models of these metaclassifiers were trained using simple average, weighted average, majority voting, dynamic weighted average, and best confidence selection methods. Among the metaclassifiers, XGB and dynamic weighted average achieved the highest accuracies of 86.83% and 87.17% respectively, with XGB also yielding the lowest FNR, highlighting its inherent capability in handling heterogeneous and diverse features. To simulate real-world emergency driving conditions, where constant sensor inputs may be intermittent, missing modalities were simulated in Google Colab during training of the metaclassifiers. Later, the robustness of the metaclassifiers was analyzed across various missing modality scenarios, where up to three sensor inputs could be unavailable. In this analysis, XGB achieved the highest stability with the lowest SD of 0.039 across all missing modality scenarios, demonstrating its ability to maintain reliable driver fatigue detection performance even when some sensor inputs are missing.

Copyright

© 2026 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Zulfa Sammadile
AU  - Deogratias Shidende
AU  - Bonny Mgawe
AU  - Sabine Moebs
PY  - 2026
DA  - 2026/06/13
TI  - Late Fusion-Based Multimodal Machine Learning for Driver Fatigue Detection in Emergency Response Driving Scenarios
BT  - Proceedings of the FIREtalk Conference - Research on FIRE! (research-on-fire 2025)
PB  - Atlantis Press
SP  - 104
EP  - 123
SN  - 2352-5398
UR  - https://doi.org/10.2991/978-94-6239-705-7_8
DO  - 10.2991/978-94-6239-705-7_8
ID  - Sammadile2026
ER  -