Modelling Generalization Under Distribution Shift for Machine Learning-Based Malware Detection

DOI

10.2991/978-94-6239-723-1_39How to use a DOI?

Keywords

Machine learning; Generalization Modelling; Malware Detection; Data Analysis

Abstract

Machine learning has become a widely used approach for malware detection due to its strong performance on benchmark datasets. However, in real-world environments malware continuously evolves through the emergence of new malware families, obfuscation techniques, and behavioural modifications, which cause changes in the underlying data distribution. Such distribution shifts can significantly affect the generalization ability of machine learning models and often create a gap between laboratory evaluation results and real-world deployment performance. This paper investigates the impact of distribution shift on malware detectors built using static executable features. A Random Forest classifier is trained using the EMBER-2018 dataset, a large-scale dataset containing static features extracted from Windows Portable Executable (PE) files. The model is evaluated under three experimental settings: standard independent and identically distributed (IID) evaluation, mild distribution shift created using temporally separated training and testing data, and strong distribution shift simulated through artificial feature corruption. Experimental results show that although the model achieves near-perfect performance under IID conditions (ROC-AUC = 0.996), its performance degrades under mild distribution shift (ROC-AUC = 0.990) and significantly collapses under strong distribution shift (ROC-AUC = 0.700). To mitigate this degradation, a noise-augmented training strategy is introduced, which improves robustness and increases the ROC-AUC to 0.780 under strong shift conditions. These findings demonstrate that benchmark accuracy alone is not a reliable indicator of real-world malware detection performance and highlight the importance of evaluating detection systems under realistic distribution shift scenarios.

Copyright

© 2026 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Prerna P. Ghorpade
AU  - Aaditya S. Dhanwate
AU  - Bhoomi B. Budhani
PY  - 2026
DA  - 2026/07/14
TI  - Modelling Generalization Under Distribution Shift for Machine Learning-Based Malware Detection
BT  - Proceedings of the International Conference on Responsible, Risk-aware, and Regulated AI (RRRAI 2026)
PB  - Atlantis Press
SP  - 440
EP  - 452
SN  - 1951-6851
UR  - https://doi.org/10.2991/978-94-6239-723-1_39
DO  - 10.2991/978-94-6239-723-1_39
ID  - Ghorpade2026
ER  -