Robust Sign Language Discrimination with Transfer Learning on Resnet And Transformer Networks

DOI

10.2991/978-94-6463-718-2_82How to use a DOI?

Keywords

Sign Language Recognition; Resnet; Transformer Networks; Transfer Learning; Spatial-Temporal Modeling; Real-Time Deployment; Gesture Recognition

Abstract

It is a crucial aid for people with hearing and speech deficiencies. However, there are no efficient recognition systems to enable continuous communication of sign language users with non-signers. This paper presents a robust sign language discrimination system based on the transfer learning of ResNet and transformer networks. ResNet is used to extract spatial features and Transformers are used to model the time dependencies between features, providing a detailed understanding of the sign language gestures. It shows the performance of a trained system in comparison with state-of-the-art models, and overall accuracy was 94.7% with significant reduction in inference time of 180 ms, and obtained results on a test environment showed significant robustness in challenging real-world scenarios like low lighting, dynamic backgrounds and partial occlusions. This is practical for deployment on resource-constrained devices due to the system’s real-time capability and scalability. Further research will address the challenges posed by occlusions and extend the system’s applicability to more underrepresented sign languages. Your training is based on data available until October 2023.

Copyright

© 2025 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - P. Vasuki
AU  - V. Vennila
AU  - V. Savitha
AU  - B. Arulmurugan
AU  - R. Deepan
AU  - P. Harish
PY  - 2025
DA  - 2025/05/23
TI  - Robust Sign Language Discrimination with Transfer Learning on Resnet And Transformer Networks
BT  - Proceedings of the International Conference on Sustainability Innovation in Computing and Engineering (ICSICE 2024)
PB  - Atlantis Press
SP  - 969
EP  - 980
SN  - 2352-538X
UR  - https://doi.org/10.2991/978-94-6463-718-2_82
DO  - 10.2991/978-94-6463-718-2_82
ID  - Vasuki2025
ER  -