Integrating Deep Learning and Computer Vision for Recognizing American Sign Language

Year : 2024 | Volume : 14 | Issue : 03 | Page : 10 17
    By

    Shivam Khachane,

  • Kanchan Pujari,

  • Rushikesh Khandagale,

  • Nishu Lodhi,

  1. Student, Department of Electronics and Telecommunication, Smt. Kashibai Navale College of Engineering, Savitribai Phule Pune University, Pune, Maharashtra, India
  2. Assistant Professor, Department of Electronics and Telecommunication, Smt. Kashibai Navale College of Engineering, Savitribai Phule Pune University, Pune, Maharashtra, India
  3. Student, Department of Electronics and Telecommunication, Smt. Kashibai Navale College of Engineering, Savitribai Phule Pune University, Pune, Maharashtra, India
  4. Student, Department of Electronics and Telecommunication, Smt. Kashibai Navale College of Engineering, Savitribai Phule Pune University, Pune, Maharashtra, India

Abstract

The only way the hearing-impaired community can exchange ideas is by utilizing non-verbal communication. The main challenge, however, is that the non-impaired community, which may not comprehend non-verbal communication, would struggle to communicate effectively with this group, and vice versa. The project is purposely devised to admit unwilling and dumb societies to transport ideas and connect with the organization. It aims to bridge the gap between the hearing- and speech-impaired community and the non-impaired community. Speech impairment is a restriction that influences an individual’s ability to write efficiently. Many existing studies have proposed procedures for non-verbal communication recognition. This study focuses on full American Sign Language (ASL) recognition. With current advances in deep learning and computer vision, significant progress has been made in the field of gesture recognition. Thus, this study aims to extract features from finger and hand motions. The system does not require the background to be completely dark and functions effectively on almost any background. The project utilizes a concept alteration plan to identify non-verbal communication, specifically English alphabet gestures, and convert them into meaningful ideas. The focus of this work is to design a fantasy-based application that provides ASL translation, facilitating communication between signers and non-signers. The proposed model processes figure sequences to extract temporal and spatial features from structured data. Subsequently, an Inception-based convolutional neural network (CNN) is employed to recognize spatial features.

Keywords: Convolutional neural network (CNN), machine learning, American Sign Language (ASL), computer vision, deep learning

[This article belongs to Current Trends in Information Technology ]

How to cite this article:
Shivam Khachane, Kanchan Pujari, Rushikesh Khandagale, Nishu Lodhi. Integrating Deep Learning and Computer Vision for Recognizing American Sign Language. Current Trends in Information Technology. 2024; 14(03):10-17.
How to cite this URL:
Shivam Khachane, Kanchan Pujari, Rushikesh Khandagale, Nishu Lodhi. Integrating Deep Learning and Computer Vision for Recognizing American Sign Language. Current Trends in Information Technology. 2024; 14(03):10-17. Available from: https://journals.stmjournals.com/ctit/article=2024/view=171688


References

  1. Bantupalli K, Xie Y. American sign language recognition using deep learning and computer vision. 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA, 2018, pp. 4896–99. p. 4896–9. DOI: 10.1109/BigData.2018.8622141.
  2. Cohen I, Sebe N, Garg A, Chen LS, Huang TS. Facial expression recognition from video sequences: Temporal and static modeling. Comput Vis Image Underst. 2003;91:160–87. DOI: 10.1016/S1077-3142(03)00081-X.
  3. Rajan RG, Leo MJ. A comprehensive analysis on sign language recognition system. Int J Recent Technol Eng. 2019;7(6):749–55.
  4. Nandy A, Prasad JS, Mondal S, Chakraborty P, Nandi GC. Recognition of isolated Indian sign language gesture in real time. In: Das VV, editor. Information processing and management. BAIP 2010. Communications in computer and information science, vol. 70. Springer: Berlin, Heidelberg; 2010. DOI: 10.1007/978-3-642-12214-9_18.
  5. Tripathi K, Baranwal N, Nandi GC. Continuous dynamic Indian sign language gesture recognition with invariant backgrounds. 2015 International Conference on Advances in Computing, Communications and Informatics (ICACCI), Kochi, India, 2015, pp. 2211–6. DOI: 10.1109/ICACCI.2015.7275945.
  6. Neidle C, Thangali A, Sclaroff S. Challenges in development of the American Sign Language Lexicon Video Dataset (ASLLVD) corpus. In: Proceedings of the 5th Workshop on the Representation and Processing of Sign Languages: Interactions between Corpus and Lexicon, Language Resources and Evaluation Conference (LREC) 2012; Istanbul, Turkey. Available from: https://hdl.handle.net/2144/31899
  7. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015, pp. 1–9. DOI: 10.1109/CVPR.2015.7298594.
  8. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9:1735–80. DOI: 10.1162/neco.1997.9.8.1735. PubMed: 9377276.
  9. Srivastava S, Gangwar A, Mishra R, Singh S. Sign language recognition system using TensorFlow object detection API. In: Woungang I, Dhurandher SK, Pattanaik KK, Verma A, Verma P, editors. Advanced network technologies and intelligent computing. ANTIC 2021. Communications in Computer and Information Science, vol 1534. Cham, Germany; 2022. p. 634–646. DOI: 10.1007/978-3-030-96040-7_48.
  10. Pugeault N, Bowden R. Spelling it out: Real-time ASL fingerspelling recognition. 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain. 2011, pp. 1114–9. DOI: 10.1109/ICCVW.2011.6130290.
Regular Issue Subscription Review Article
Volume 14
Issue 03
Received 03/07/2024
Accepted 05/09/2024
Published 11/09/2024
507


My IP

PlumX Metrics