Multi-modal Sentiment Analysis: Addressing Modality Loss and Alignment Issues

DOI

10.2991/978-94-6463-823-3_96How to use a DOI?

Keywords

Multimodal Sentiment; Modality loss; Modality Alignment

Abstract

With the rapid development of artificial intelligence technology, multimodal sentiment analysis has gradually become a research focus. This paper delves into the primary challenges faced in practical applications: modality absence and alignment issues, summarizing a series of technical methods and solutions. Firstly, it outlines the impacts of modality loss, including significant degradation in model performance due to sensor failures, network transmission delays, or privacy protections. To address these issues, researchers have proposed modal compensation mechanisms, Single Branch Model Invariant Learning (SRMM), Generative Adversarial Network (GANs), and contrastive learning methods to enhance models’ robustness against missing modalities. Secondly, concerning modality alignment, this paper discusses approaches based on attention mechanisms like Transformers, shared representation learning strategies such as Variational Autoencoders (VAEs), and graph-based structures using Graph Convolutional Networks (GCNs). Despite significant advancements in processing multimodal data, these methods still face challenges such as low computational efficiency, strong dependence on data quality, and sensitivity to noise. The paper emphasizes that future research should explore efficient training methods, balancing theoretical innovation, engineering optimization, aiming to build an efficient, robust, and trustworthy technical system.

Copyright

© 2025 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Ipshing Liu
AU  - Yuyang Peng
AU  - Jianing Zhang
PY  - 2025
DA  - 2025/08/31
TI  - Multi-modal Sentiment Analysis: Addressing Modality Loss and Alignment Issues
BT  - Proceedings of the 2025 3rd International Conference on Image, Algorithms, and Artificial Intelligence (ICIAAI 2025)
PB  - Atlantis Press
SP  - 980
EP  - 992
SN  - 2352-538X
UR  - https://doi.org/10.2991/978-94-6463-823-3_96
DO  - 10.2991/978-94-6463-823-3_96
ID  - Liu2025
ER  -