Proceedings of the 6th International Conference on Deep Learning, Artificial Intelligence and Robotics (ICDLAIR 2024)

<Previous Article In Volume
Next Article In Volume>

Indian Sign Language Communication Using LSTM, MediaPipe Holisitic and Llama

Authors
Pratham Shah1, *, Arjun Pareek1, *, Kanika Chitnis1, *, Vishakha Shelke1, Swapnil Gharat1, Sacchit Wathe1
1DJSCE, Mumbai, India
*Corresponding author. Email: pratham0925@gmail.com
*Corresponding author. Email: arjunpareek03@gmail.com
*Corresponding author. Email: chitniskanika@gmail.com
Corresponding Authors
Pratham Shah, Arjun Pareek, Kanika Chitnis
Available Online 25 June 2025.
DOI
10.2991/978-94-6463-740-3_21How to use a DOI?
Keywords
Sign Language Recognition (SLR); Indian Sign Language (ISL); Long-Short Term Memory (LSTM); MediaPipe Holistic; LLaMA 3
Abstract

This paper focuses on the translation of Indian Sign Language (ISL) into text, to expand the range of people with whom individuals with hearing impairments can communicate. The proposed method for this utilizes Mediapipe Holistic for capturing hand gestures and Long Short-Term Memory or LSTM networks are used to model the temporal dependencies in the gesture sequences. The novelty of this research not only includes a custom-made dataset to train the model to achieve high recognition accuracy, but also the use of LLaMa 3. to form a meaningful sentence from the detected words. The results prove that the gestures are recognized as their corresponding words with an accuracy of 92.36% and a precision of 92.76%. Compared to existing ISL translation systems, our method integrates Llama3 for sentence formation, improving coherence and meaning in generated text. This solution will cater to enhancing accessibility in public services, and day-to-day communication, bridging the gap between hearing-impaired individuals and the broader community.

Copyright
© 2025 The Author(s)
Open Access
Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

<Previous Article In Volume
Next Article In Volume>
Volume Title
Proceedings of the 6th International Conference on Deep Learning, Artificial Intelligence and Robotics (ICDLAIR 2024)
Series
Advances in Intelligent Systems Research
Publication Date
25 June 2025
ISBN
978-94-6463-740-3
ISSN
1951-6851
DOI
10.2991/978-94-6463-740-3_21How to use a DOI?
Copyright
© 2025 The Author(s)
Open Access
Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Cite this article

risenwbib
TY  - CONF
AU  - Pratham Shah
AU  - Arjun Pareek
AU  - Kanika Chitnis
AU  - Vishakha Shelke
AU  - Swapnil Gharat
AU  - Sacchit Wathe
PY  - 2025
DA  - 2025/06/25
TI  - Indian Sign Language Communication Using LSTM, MediaPipe Holisitic and Llama
BT  - Proceedings of the 6th International Conference on Deep Learning, Artificial Intelligence and Robotics (ICDLAIR 2024)
PB  - Atlantis Press
SP  - 240
EP  - 250
SN  - 1951-6851
UR  - https://doi.org/10.2991/978-94-6463-740-3_21
DO  - 10.2991/978-94-6463-740-3_21
ID  - Shah2025
ER  -