Abstract: The existing gesture-based operating systems can only simply operate a single piece of software or a specific system, and are not compatible with other applications of mainstream operating systems. In this paper, based on the MediaPipe gesture recognition framework, we design HandMouse, a virtual mouse system that operates using hand gestures. It has the following characteristics: 1. The user does not have contact with the computer hardware when using the system; 2. It requires only one hand to operate, and the design of the gesture considers the ergonomics of the hand; 3. It has most of the functions commonly used in a physical mouse; 4. It can locate the operating area with relative precision. We invited 27 participants to use and evaluate the virtual mouse and then conducted an experiment to compare the performance of the virtual mouse with the physical mouse. The results show that this virtual mouse has a good learning effect and is a great alternative to the physical mouse in public places. The demonstrated operation video is available on https://github.com/wanzhuxie/HandMouse-IJACSA.

Qiujiao Wang and Zhijie Xie, “Replace Your Mouse with Your Hand! HandMouse: A Gesture-Based Virtual Mouse System” International Journal of Advanced Computer Science and Applications(IJACSA), 15(11), 2024. http://dx.doi.org/10.14569/IJACSA.2024.0151163

@article{Wang2024,
title = {Replace Your Mouse with Your Hand! HandMouse: A Gesture-Based Virtual Mouse System},
journal = {International Journal of Advanced Computer Science and Applications},
doi = {10.14569/IJACSA.2024.0151163},
url = {http://dx.doi.org/10.14569/IJACSA.2024.0151163},
year = {2024},
publisher = {The Science and Information Organization},
volume = {15},
number = {11},
author = {Qiujiao Wang and Zhijie Xie}
}