Siddhesh Sachin Kapure,
Paras Sharad Kasar,
Rituja Satish Pawar,
Naresh Vasudev Sathe,
Dipti D. Survase,
- Student, Department of Artificial Intelligence & Data Science, Jhulelal Institute of Technology, Maharashtra, India
- Student, Department of Artificial Intelligence & Data Science, Jhulelal Institute of Technology, Maharashtra, India
- Student, Department of Artificial Intelligence & Data Science, Jhulelal Institute of Technology, Maharashtra, India
- Student, Department of Artificial Intelligence & Data Science, Jhulelal Institute of Technology, Maharashtra, India
- Professor, Department of Artificial Intelligence & Data Science, Jhulelal Institute of Technology, Maharashtra, India
Abstract
Communication is a critical part of life, and effective communication between two or more people can sometimes be challenging due to a person’s inability to communicate because of a disability such as vision loss, hearing loss, or speech loss. In the case of individuals with disabilities, they are usually given time to send out messages and receive messages, meaning they need tools that only help them with one area of communication at a time. This paper introduces a new way to communicate with the help of artificial intelligence (AI) and the Internet of Things (IoT) for individuals who have sensory impairments in order to communicate better with each other and to find a way to create a “bridge” for persons with disabilities to understand each other. For example, the system would allow an individual to see gestures, and convert those gestures to text, return the text to the person that made the gesture in the form of speech or Braille, and provide a way to assist an individual with a sensory impairment to move about safely by identifying obstacles in the way and indicating how far away the obstacle is from the person. The system uses various tools, such as MediaPipe, to track the movement of the hands and other tools to convert the gesture to text and vice versa, and communicates the information to individuals around them. This new communication system can be made available to many users and is an affordable way for individuals to better communicate with one another. As such, this new technology applies to day-to-day activities to assist persons with sensory impairments to live more independently and be more involved in their communities by providing access to technology that has previously been unavailable to them. The proposed new Communication System for Individuals with Sensory Impairments is a work in progress and will likely take time to complete.
Keywords: Assistive technology, gesture recognition, speech processing, YOLO, Braille, AI, IoT
[This article belongs to International Journal of Robotics and Automation in Mechanics ]
References
- Zhang Y, Wang Y, Li F, Yu W, Wang C, Jiang Y. Sign language recognition based on CNN-BiLSTM using RF signals. IEEE Access. 2024 Dec 13;12:190487-504.
- Wang Z, Li D, Jiang R, Okumura M. Continuous sign language recognition with multi-scale spatial-temporal feature enhancement. IEEE Access. 2025 Jan 6;13:5491–506..
- Tao T, Zhao Y, Liu T, Zhu J. Sign language recognition: A comprehensive review of traditional and deep learning approaches, datasets, and challenges. Ieee Access. 2024 May 8;12:75034–60.
- Weerasinghe RL, Ganegoda GU. A Comprehensive Review on Vision-based Sign Language Detection and Recognition. In2022 International Research Conference on Smart Computing and Systems Engineering (SCSE) 2022 Sep 1 (Vol. 5, pp. 88–95). IEEE.
- Sridhar B, Saivishnu G, ManiShanker V, Lakshmi DD, Hariharan S. Summarization of video into text and text to braille script. In2024 International Conference on Knowledge Engineering and Communication Systems (ICKECS) 2024 Apr 18 (Vol. 1, pp. 1–5). IEEE..
- Alosail D, Aldolah H, Alabdulwahab L, Bashar A, Khan M. Smart glove for bi-lingual sign language recognition using machine learning. In2023 International Conference on Intelligent Data Communication Technologies and Internet of Things (IDCIoT) 2023 Jan 5 (pp. 409–415). IEEE.
- Sharma J, Gill KS, Kumar M, Rawat R. Gesture to text recognition using deep learning approach with VGG16 and ResNet50 for sign language. In2024 4th Asian Conference on Innovation in Technology (ASIANCON) 2024 Aug 23 (pp. 1–5). IEEE..
- Zhang J, Wang Q, Wang Q. HDTSLR: A framework based on hierarchical dynamic positional encoding for sign language recognition. IEEE Transactions on Mobile Computing. 2023 Aug 31;23(5):5631–43.
- Sharma S, Sreemathy R, Turuk M, Jagdale J, Khurana S. Real-time word level sign language recognition using YOLOv4. In2022 International Conference on Futuristic Technologies (INCOFT) 2022 Nov 25 (pp. 1–7). IEEE.
- Antad SM, Chakrabarty S, Bhat S, Bisen S, Jain S. Sign language translation across multiple languages. In2024 International Conference on Emerging Systems and Intelligent Computing (ESIC) 2024 Feb 9 (pp. 741–746). IEEE.
- Nehra T, Saisanthiya D, Modi A. Indian sign language (isl) recognition and translation using mediapipe and lstm. In2023 World Conference on Communication & Computing (WCONF) 2023 Jul 14 (pp. 1–5). IEEE.
- Verma I, Rai A, Yadav H, Rastogi V, Satija S. Video to braille transcription for visually impaired people. In2021 International Conference on Simulation, Automation & Smart Manufacturing (SASM) 2021 Aug 20 (pp. 1–3). IEEE.
| Volume | 04 | |
| Issue | 02 | |
| Received | 12/05/2026 | |
| Accepted | 11/07/2026 | |
| Published | 09/10/2026 | |
| Publication Time | 150 Days |