Hybrid Deep Learning Framework for Multimodal Deceptive Behavior Detection Using Micro-Expressions and Physiological Signals

DOI

10.2991/978-94-6463-718-2_60How to use a DOI?

Keywords

Hybrid Deep Learning; Deception Detection; Multimodal Fusion; Micro-Expressions; Physiological Signals; CNN; LSTM; Real-Time Detection

Abstract

Existing research on deception detection focuses on behavioral and psychological approaches; however, these methods have limitations in accuracy and are labor-intensive, making them unsuitable for practical applications. Our framework combines video, audio, and physiological signals to facilitate a more holistic view of both emotional and physical cues during conversations, leading to advanced recognition of subtle and often missed indicators of deception. This ability to learn from multiple modalities provides us with a clear advantage over traditional methods that operate only via a single modality, offering a significant improvement in dealing with noisy or missing data which is quite common in real-world applications. Small movements and physiological reactions, e.g., heart rate, skin conductance, are clearly useful for improved detection, which is an advantage of the proposed system. Furthermore, leveraging such advanced deep learning techniques (e.g., CNN and LSTM model) guarantees that temporal dependencies and non-linear feature interactions are well learnt over time leading to non-invasive and real-time monitoring of deception detection process. Also, it is efficient and scalable, making it a useful approach for security, interrogation, and behavior applications.

Copyright

© 2025 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - S. Savitha
AU  - R. Keerthana
AU  - P. Vasuki
AU  - M. Baranidharan
AU  - M. Barathraj
AU  - G. T. Hareesh
PY  - 2025
DA  - 2025/05/23
TI  - Hybrid Deep Learning Framework for Multimodal Deceptive Behavior Detection Using Micro-Expressions and Physiological Signals
BT  - Proceedings of the International Conference on Sustainability Innovation in Computing and Engineering (ICSICE 2024)
PB  - Atlantis Press
SP  - 698
EP  - 709
SN  - 2352-538X
UR  - https://doi.org/10.2991/978-94-6463-718-2_60
DO  - 10.2991/978-94-6463-718-2_60
ID  - Savitha2025
ER  -