Poem-to-Music Retrieval through Multilingual Emotion Curves: A Low-Resource and Explainable Approach

DOI

10.2991/978-94-6463-978-0_24How to use a DOI?

Keywords

cross-lingual embeddings; emotion lexicon; poetry–music alignment; multi-modal AI; low-resource languages

Abstract

Poetry and music share deep emotional connections, yet their computational alignment has remained largely unexplored. Existing music–text retrieval methods typically focus on English prompts or largescale audio–caption datasets, leaving low-resource languages and creative genres such as poetry understudied. This work introduces a lightweight and interpretable pipeline that aligns lines of Hindi and Kannada poems with music captions from the MusicCaps dataset. Our approach combines multilingual sentence embeddings with emotion lexicons to build an emotion curve across poem lines, which then guides caption retrieval. A baseline reuses a single caption for all lines, whereas our method produces line-specific captions enriched with emotional and instrumental cues. Quantitative evaluation shows consistent improvements over baseline (∆≈ +0.14 for Hindi, ∆≈ +0.19 for Kannada, both with statistically significant gains), while qualitative analysis highlights a smoother alignment of emotion and musical style. The results suggest that even small lexicons and compact multilingual models can support poetry-to music retrieval in low-resource settings, paving the way for cross-lingual creative AI applications while offering a low-resource, explainable foundation that can later be extended to audio-based alignment.

Copyright

© 2025 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Poornima Shetty
AU  - S. N. Muralikrishna
AU  - V. S. Shrishma Rao
AU  - Aruna Doreen Manezes
PY  - 2025
DA  - 2025/12/31
TI  - Poem-to-Music Retrieval through Multilingual Emotion Curves: A Low-Resource and Explainable Approach
BT  - Proceedings of the 1st Engineering Data Analytics and Management Conference (EAMCON 2025)
PB  - Atlantis Press
SP  - 257
EP  - 268
SN  - 2352-5401
UR  - https://doi.org/10.2991/978-94-6463-978-0_24
DO  - 10.2991/978-94-6463-978-0_24
ID  - Shetty2025
ER  -