Multimodal Machine Learning Framework Combining EEG and Physiological Signals for Mental Health Diagnosis: A Review

DOI

10.2991/978-94-6239-628-9_32How to use a DOI?

Keywords

Multimodal machine learning framework; Electroencephalogram (EEG); physiological signals; Mental health; diagnosis; feature extraction; Convolutional Neural Networks (CNN); Support Vector Machines (SVM); early detection

Abstract

The prevalence of mental health issues like anxiety and depression is increasing globally. An early detection can play a major role in improving treatment results. Electroencephalogram (EEG) and physiological signals are combined in multimodal machine learning frameworks to provide a strong and reliable approach for diagnosing and predicting mental health condition accurately. While physiological signals such as Heart Rate Variability (HRV), Galvanic Skin Response (GSR), and electrocardiogram (ECG) show how the body reacts to emotional states, EEG represents brain activity. A comprehensive view of both physical and mental health can be achieved by integrating these modalities. When physiological signals are combined with EEG, machine learning improves multimodal diagnosis by efficiently capturing emotional and cognitive patterns. The study shows that best networks for accurate diagnosis are SVM, CNN, and hybrid networks. This framework outperforms single-modality approaches by using classifiers like CNN and SVM, as well as time-frequency and nonlinear statistical methods for feature extraction. Effective stress predictors include physiological indicators with high explanatory power and ease of collection, such as skin response and heart rate. Study indicates that combining body and brain signals enhances diagnostic accuracy and supports objective, early detection of mental health conditions.

Copyright

© 2026 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Pijush Kanti Ghosh
AU  - Rakesh Kumar
AU  - Momita Kumari
AU  - Nishant Goyal
AU  - Shalini Mahato
PY  - 2026
DA  - 2026/03/31
TI  - Multimodal Machine Learning Framework Combining EEG and Physiological Signals for Mental Health Diagnosis: A Review
BT  - Proceedings of the International Conference on Recent Trends in Intelligent Computing, Manufacturing, and Electronics (rTIME 2025)
PB  - Atlantis Press
SP  - 357
EP  - 368
SN  - 2352-5401
UR  - https://doi.org/10.2991/978-94-6239-628-9_32
DO  - 10.2991/978-94-6239-628-9_32
ID  - Ghosh2026
ER  -