Multimodal Emotion Detection: An Integrated Approach to Understanding Human Emotions

DOI

10.2991/978-94-6463-738-0_90How to use a DOI?

Keywords

Emotion Detection; RoBERTa; Convolutional Neural Network; Multimodal Analysis; web application; Real-time Processing; Emotional Intelligence; annotation Tool

Abstract

The Emotion Detection web application is a powerful tool designed to analyze and interpret human emotions from diverse data inputs, such as text, images, video files, and live video feeds. It leverages a fine-tuned RoBERTa model for emotion analysis in text and a custom convolutional neural network (CNN) for visual inputs, achieving an accuracy rate of 93%. The web application also serves as an annotation tool, allowing users to label emotional content and tone in text, whereas in images, videos, and live cams, the emotional expressions are detected and labelled. This feature enhances its applicability in areas such as mental health monitoring, interactive storytelling, customer behaviour analysis. By integrating multimodal analysis, the application provides enriched visualizations and insights into emotional dynamics, facilitating deeper human-computer interaction. While it supports real-time processing for text and images, video annotation requires the analysis of individual frames, which can slow down processing times, especially for larger video files. The system’s capabilities in real-time emotion detection and multimodal analysis demonstrate its transformative potential in improving emotional understanding and enhancing digital interactions across a wide range of fields. The application’s versatility and deep learning approach showcase its significant impact on advancing emotional intelligence in digital environments.

Copyright

© 2025 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - R. Jagadeesh
AU  - M. Babu
PY  - 2025
DA  - 2025/06/22
TI  - Multimodal Emotion Detection: An Integrated Approach to Understanding Human Emotions
BT  - Proceedings of the International Conference on Advances and Applications in Artificial Intelligence (ICAAAI 2025)
PB  - Atlantis Press
SP  - 1174
EP  - 1188
SN  - 1951-6851
UR  - https://doi.org/10.2991/978-94-6463-738-0_90
DO  - 10.2991/978-94-6463-738-0_90
ID  - Jagadeesh2025
ER  -