Hand-Tracking-Based Mouse Control for Touchless Human-Computer Interaction: Feasibility and Future Enhancement

Year : 2025 | Volume : 13 | Issue : 02 | Page : 18 25
    By

    Sameer Awasthi,

  • Amarnath,

  • Abhay Dixit,

  • Arpit Singh,

  1. Head of Department, Computer Science Engineering in Artificial Intelligence and Machine Learning, Bansal Institute of Engineering & Technology, Lucknow, Uttar Pradesh, India
  2. Student, Computer Science Engineering in Artificial Intelligence and Machine Learning, Bansal Institute of Engineering & Technology, Lucknow, Uttar Pradesh, India
  3. Student, Computer Science Engineering in Artificial Intelligence and Machine Learning, Bansal Institute of Engineering & Technology, Lucknow, Uttar Pradesh, India
  4. Student, Computer Science Engineering in Artificial Intelligence and Machine Learning, Bansal Institute of Engineering & Technology, Lucknow, Uttar Pradesh, India

Abstract

This paper presents a novel approach to human-computer interaction through a hand-tracking-based mouse control system using computer vision. The system utilizes OpenCV, MediaPipe, and the Cvzone Hand Tracking Module to identify and monitor hand movements in real-time. By utilizing hand landmarks, the system maps finger positions to cursor movement and enables essential functions such as clicking, scrolling, and double-clicking. The primary motivation for developing this system is to create a touchless alternative to traditional input devices, offering enhanced accessibility and usability. The methodology involves capturing video input via a webcam, detecting hands using MediaPipe’s pre-trained models, and implementing a gesture recognition algorithm to interpret user actions. Cursor movements are mapped through an interpolation function, while clicking and scrolling gestures are detected based on finger positions and distances between key landmarks. Performance evaluation indicates an average response time of 20 to 30 ms, ensuring real-time usability. The system achieves an accuracy rate of 95% in controlled lighting conditions. However, challenges such as lighting variations and occlusion remain key areas for improvement. To mitigate unintended gestures, delays and threading mechanisms are incorporated. Future enhancements will focus on improving gesture classification with artificial intelligence models, addressing lighting-related limitations, and incorporating multi-hand interactions for increased functionality. This research contributes to the field of gesture-based computing, offering a cost-effective, hardware-independent solution for modern human-computer interaction.

Keywords: Hand tracking, computer vision, gesture recognition, virtual mouse, human-computer interaction (HCI)

[This article belongs to Research & Reviews: A Journal of Embedded System & Applications ]

How to cite this article:
Sameer Awasthi, Amarnath, Abhay Dixit, Arpit Singh. Hand-Tracking-Based Mouse Control for Touchless Human-Computer Interaction: Feasibility and Future Enhancement. Research & Reviews: A Journal of Embedded System & Applications. 2025; 13(02):18-25.
How to cite this URL:
Sameer Awasthi, Amarnath, Abhay Dixit, Arpit Singh. Hand-Tracking-Based Mouse Control for Touchless Human-Computer Interaction: Feasibility and Future Enhancement. Research & Reviews: A Journal of Embedded System & Applications. 2025; 13(02):18-25. Available from: https://journals.stmjournals.com/rrjoesa/article=2025/view=208533


References

  1. Mohamed N, Mustafa MB, Jomhari N. A review of the hand gesture recognition system: Current progress and future directions. IEEE Access. 2021;9:157422–36. doi:10.1109/ACCESS.2021. 3129650.
  2. Li J, Liu X, Wang Z, Zhang T, Qiu S, Zhao H, et al. Real-time hand gesture tracking for human– computer interface based on multi-sensor data fusion. IEEE Sens J. 2021;21:26642–54. doi:10.1109/JSEN.2021.3122236.
  3. Samkari E, Arif M, Alghamdi M, Al Ghamdi MA. Human pose estimation using deep learning: A systematic literature review. Mach Learn Knowl Extr. 2023;5:1612–59. doi:10.3390/make5040081.
  4. Shah P, Patel M, Shah H, Patel P, Joshi S, Darji M, et al. Exploration of historical and modern perspective on hand gesture recognition with AI and ML. In: Tuba M, Akashe S, Joshi A, editors. ICT Systems and Sustainability. ICT4SD 2024. Lecture Notes in Networks and Systems, vol 1194. Singapore: Springer; 2025. p. 315–25. doi:10.1007/978-981-97-9523-9_27.
  5. Kukreja S, Singh A, Chauhan V. Hand gesture vision: Integrating computer vision and machine learning for enhanced communication. 2024 1st International Conference on Advances in Computing, Communication and Networking (ICAC2N), Greater Noida, India. 2024. pp. 35-40. doi: 10.1109/ICAC2N63387.2024.10895070.
  6.  Kim J, Jang H, Kim D, Lee J. Exploration of the virtual reality teleportation methods using hand- tracking, eye-tracking, and EEG. Int J Hum Comput Interact. 2023;39:4112–25. doi:10.1080/ 10447318.2022.2109248.
  7. Lei Y, He S, Khamis M, Ye J. An end-to-end review of gaze estimation and its interactive applications on handheld mobile devices. ACM Comput Surv. 2024;56:1–38. doi:10.1145/3606947.
  8. Alonazi M, Ansar H, Al Mudawi NA, Alotaibi SS, Almujally NA, Alazeb A, et al. Smart healthcare hand gesture recognition using CNN-based detector and deep belief network. IEEE Access. 2023;11:84922–33. doi:10.1109/ACCESS.2023.3289389.
  9. Rehman IU, Ullah S, Ali N, Rabbi I, Khan RU. Gesture-based guidance for navigation in virtual environments. J Multimodal User Interfaces. 2022;16:371–83. doi:10.1007/s12193-022-00395-1.
  10. Ai Q, Zhao M, Chen K, Liu Q, He D, Li Z, et al. Development and application of graphene sensors in human–computer interaction: A review. IEEE Sens J. 2024;24:7406–19. doi:10.1109/JSEN. 2024.3360457.
  11. Qi J, Ma L, Cui Z, Yu Y. Computer vision-based hand gesture recognition for human–robot interaction: A review. Complex Intell Syst. 2024;10:1581–606. doi:10.1007/s40747-023-01173-6.
  12. Ojeda-Castelo JJ, Capobianco-Uriarte MLM, Piedra-Fernandez JA, Ayala R. A survey on intelligent gesture recognition techniques. IEEE Access. 2022;10:87135–56. doi:10.1109/ACCESS.2022. 3199358.
  13. Wisener WJ, Rodriguez JD, Ovando A, Woolford C, Patel K. A top-view hand gesture recognition system for IoT applications. 2023 5th International Conference on Smart Systems and Inventive Technology (ICSSIT), Tirunelveli, India. 2023, pp. 430–4. doi:10.1109/ICSSIT55814.2023. 10060969.
  14. Ruocco M, Duggan J, Jaiswal C, O’Neill B, Majeski K. Leveraging AI face-tracking and gesture recognition for hands-free computing: Bridging the gap for users with physical disabilities. 2024 IEEE Global Humanitarian Technology Conference (GHTC), Radnor, PA, USA. 2024. pp. 232–9. doi:10.1109/GHTC62424.2024.10771580.
  15. Mohd Dhuzuki NH, Zainuddin AA, Kamarul Zaman NAS, Ahmad Razmi ANM, Kaitane WS, Ahmad Puzi A, et al. Design and implementation of a deep learning-based hand gesture recognition system for rehabilitation Internet-of-things (RIoT) environments using MediaPipe. IIUM Eng J. 2025;26:353–72. doi:10.31436/iiumej.v26i1.3455.
  16. Dang TL, Nguyen HT, Dao DM, Nguyen HV, Luong DL, Nguyen BT, et al. SHAPE: A dataset for hand gesture recognition. Neural Comput Appl. 2022;34:21849–62. doi:10.1007/s00521-022- 07651-1.
  17. Wu D, Yang Z, Zhang P, Wang R, Yang B, Ma X. Virtual-reality interpromotion technology for metaverse: A survey. IEEE Internet Things J. 2023;10:15788–809. doi:10.1109/JIOT.2023.3265848
Regular Issue Subscription Review Article
Volume 13
Issue 02
Received 04/03/2025
Accepted 04/04/2025
Published 22/04/2025
Publication Time 49 Days
211


My IP

PlumX Metrics