Hands-Free Human–Computer Interaction for Smarter Workspaces

Year : 2026 | Volume : 13 | Issue : 02 | Page : 50 56
By

Prerana Balip,

Dnyaneshwar Sonavane,

Shaila Pawar,

Omkar Shinde,

Ayush Deoghare,

Poonam Potraje,

  1. Student, Department of Information Technology, A.C. Patil College of Engineering, Maharashtra, India
  2. Student, Department of Information Technology, A.C. Patil College of Engineering, Maharashtra, India
  3. Assistant Professor, Department of Information Technology, A.C. Patil College of Engineering, Maharashtra, India
  4. Student, Department of Information Technology, A.C. Patil College of Engineering, Maharashtra, India
  5. Student, Department of Information Technology, A.C. Patil College of Engineering, Maharashtra, India
  6. Assistant Professor, Department of Information Technology, A.C. Patil College of Engineering, Maharashtra, India

Abstract

In current digital work environments, frequent interaction with keyboards and mouse devices can interrupt workflow, especially during presentations, collaborative sessions, and multitasking activities. This study introduces a touchless human–computer interaction system that allows users to operate computer functions through hand gestures and voice instructions, minimizing the need for traditional input devices such as keyboards and mice. The proposed system integrates gesture recognition, speech processing, and optical character recognition (OCR) technologies to support a variety of interactive tasks such as slide navigation, virtual drawing, handwriting capture, and text extraction from visual input. A voice-based assistant is incorporated to execute system-level operations and respond to user commands efficiently, while a vision-based module performs real-time hand tracking and gesture detection for seamless interaction. The system is designed to improve usability, accessibility, and interaction efficiency in environments where touch-free operation is beneficial, including smart classrooms, meetings, and presentation settings. By combining computer vision and voice interaction techniques, the proposed model offers a more intuitive and flexible method of human–computer communication. Experimental results indicate that the system operates efficiently in controlled environments and can carry out various functions with high accuracy while requiring very little physical effort from the user. The project also provides scope for future enhancement in dynamic environments through improved gesture robustness, adaptive learning, and advanced AI-based interaction capabilities.

Keywords: Gesture recognition, voice assistant, human–computer interaction, virtual assistant, speech recognition

[This article belongs to Journal of Advancements in Robotics ]

How to cite this article: Prerana Balip, Dnyaneshwar Sonavane, Shaila Pawar, Omkar Shinde, Ayush Deoghare, Poonam Potraje. Hands-Free Human–Computer Interaction for Smarter Workspaces. Journal of Advancements in Robotics. 2026; 13(02):50-56.
How to cite this URL: Prerana Balip, Dnyaneshwar Sonavane, Shaila Pawar, Omkar Shinde, Ayush Deoghare, Poonam Potraje. Hands-Free Human–Computer Interaction for Smarter Workspaces. Journal of Advancements in Robotics. 2026; 13(02):50-56. Available from: https://journals.stmjournals.com/joarb/article=2026/view=259297

References

  1. Sharma P, Sharma N. Gesture recognition system. In: 2019 4th International Conference on Internet of Things: Smart Innovation and Usages (IoT-SIU); 2019; Ghaziabad, India. p. 1-3. doi:10.1109/IoT-SIU.2019.8777487.
  2. Dias PM, Jayakody K. Virtual assistant in native language. In: 2020 IEEE Asia-Pacific Conference on Geoscience, Electronics and Remote Sensing Technology (AGERS); 2020. p. 16-18. doi:10.1109/AGERS51788.2020.9452751.
  3. Pustode B, Pawar V, Pawar V, Pawar T, Pokale S. Smart presentation system using hand gestures [Preprint]. Research Square. 2023. doi:10.21203/rs.3.rs-2549833/v1.
  4. Umapathi N, Karthick G, Venkateswaran N, Jegadeesan R, Srinivas D. Desktop’s virtual assistant using Python. Eur Chem Bull. 2023;12(S3):5975-5984. doi:10.31838/ecb/2023.12.s3.667.
  5. Soroni F, Sajid SA, Bhuiyan MNH, Iqbal J, Khan MM. Hand gesture based virtual blackboard using webcam. In: 2021 IEEE 12th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON); 2021. p. 134-140. doi:10.1109/IEMCON53756.2021.9623181.
  6. Singh S, Shukla H, Sharma K, Tyagi H, Prasad J. Digitized interaction: a gesture-controlled whiteboard system with OpenCV, MediaPipe and NumPy. IPEC J Sci Technol. 2023;2(1):43-49.
  7. Idrees M, Ahmad A, Butt MA, Danish HM. Controlling PowerPoint using hand gestures in Python. Webology. 2021;18(6):1372-1388.
  8. Powar S, Kadam S, Malage S, Shingane P. Automated digital presentation control using hand gesture technique. ITM Web Conf. 2022;44:03031. doi:10.1051/itmconf/20224403031.
  9. Abid Siddique RV, Naik S. A survey on gesture control techniques for smart object interaction in disability support. Electronics. 2023;12:512.
  10. Mohamed N, Mustafa MB, Jomhari N. A review of the hand gesture recognition system: current progress and future directions. IEEE Access. 2021;9:157422-157436. doi:10.1109/ACCESS.2021.3129650.
  11. Prasetya DD, Ashar M. Design of interactive whiteboard to support e-learning. In: Proceedings of the 1st International Conference on Vocational Education and Training (ICOVET 2017); 2017 Nov 4-5; Malang, Indonesia. Paris: Atlantis Press; 2017. p. 121-124. doi:10.2991/icovet-17.2017.26.
Regular Issue Subscription Original Research
Volume 13
Issue 02
Received 08/05/2026
Accepted 04/06/2026
Published 09/06/2026
Publication Time 32 Days

Login

My IP

PlumX Metrics

Support