GestureTalk: A Real-Time CNN and LSTM Based Framework for Two-Way Indian Sign Language Communication

DOI

10.2991/978-94-6239-616-6_64How to use a DOI?

Keywords

Indian Sign Language (ISL); Gesture Recognition; Dynamic Gestures; Mobile Application; Real-Time Translation; Deep Learning; Text-to-Gesture; Voice-to-Gesture

Abstract

“Communication barriers remain a significant challenge for individuals with hearing impairments, especially in everyday face-to-face situations. To address this issue, GestureTalk has been developed as a dual-mode mobile and web application that enables real-time interpretation between Indian Sign Language (ISL) and spoken or written language. The system employs a live video stream to track hand movements using MediaPipe Hands for landmark extraction, followed by a hybrid Convolutional Neural Network–Long Short-Term Memory (CNN–LSTM) model to interpret dynamic gesture sequences. The proposed model achieves an accuracy of 97.92% with an average inference time of 35 ms per gesture on mobile devices, surpassing benchmarks such as VGG16 (95.56%) and AlexNet (93.90%). Conversely, text or speech input is rendered into animated ISL gestures, enabling smooth two-way visual communication with a response time of 40 ms. By integrating gesture recognition into video calls through WebRTC, GestureTalk delivers low-latency, secure, and accessible communication. The framework demonstrates high precision (96.85%), recall (98.87%), and F1-score (95.86%), indicating its effectiveness as an inclusive assistive technology for the hearing-impaired community.”

Copyright

© 2026 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - S. Rajalakshmi
AU  - Pratyusha Kumar Pati
AU  - K. Patchaivalliammal
AU  - S. Sreepadh Krishnan
PY  - 2026
DA  - 2026/03/31
TI  - GestureTalk: A Real-Time CNN and LSTM Based Framework for Two-Way Indian Sign Language Communication
BT  - Proceedings of the International Conference on Artificial Intelligence and Secure Data Analytics (ICAISDA 2025)
PB  - Atlantis Press
SP  - 848
EP  - 871
SN  - 1951-6851
UR  - https://doi.org/10.2991/978-94-6239-616-6_64
DO  - 10.2991/978-94-6239-616-6_64
ID  - Rajalakshmi2026
ER  -