Beta
406983

Virtual Keyboard-Mouse in Real-Time using Hand Gesture and Voice Assistant

Article

Last updated: 08 Feb 2025

Subjects

-

Tags

Image & speech Processing

Abstract

The advancement of computer applications has increasingly facilitated everyday tasks, with recent innovations focusing on voice assistants and virtual input devices. This technology is particularly beneficial for individuals with mobility challenges or in situations where direct manual computer interaction is limited. Leveraging computer vision and artificial intelligence, these applications can interpret visual data, such as human movements, and make decisions to execute corresponding commands. Our desktop application incorporates a voice assistant, virtual mouse, and virtual keyboard to enhance accessibility and usability, especially for individuals with physical disabilities or those who prefer alternative input methods. Using Python, MediaPipe, and OpenCV, the application effectively processes and interprets user gestures, providing a responsive and efficient computing experience. MediaPipe's features specifically contribute to the model's precision, optimizing hand tracking and gesture recognition for AI-driven tasks. Users can control the computer cursor through various hand gestures, type on a virtual keyboard using colored caps or tapes, and perform essential actions like left-clicking and dragging items. This integrated solution aims to improve productivity, making computers more accessible and enhancing the overall digital experience for users. The fusion of AI and computer vision in such applications continues to drive forward innovative and inclusive computing solutions, promising a future of greater accessibility and convenience in human-computer interaction.

DOI

10.21608/ASC.2025.333048.1025

Keywords

Hand gesture recognition, virtual keyboard, virtual mouse, Voice Assistant

Authors

First Name

Shaimaa

Last Name

Othman

MiddleName

-

Affiliation

Higher Institute of Computers and Information Technology, Computer Depart., El. Shorouk Academy, Cairo, Egypt

Email

dr.shimaa.osman@sha.edu.eg

City

-

Orcid

-

First Name

Hany

Last Name

Maher Sayed Lala

MiddleName

-

Affiliation

Department of Mathematics, Faculty of science, Al-Azhar University, Cairo, Egypt

Email

hmsayed@azhar.edu.eg

City

-

Orcid

-

First Name

Yosr

Last Name

Mansour

MiddleName

-

Affiliation

Higher Institute of Computers and Information Technology, Computer Depart., El. Shorouk Academy, Cairo, Egypt

Email

yosr.mansour@sha.edu.eg

City

-

Orcid

-

Volume

15

Article Issue

1

Related Issue

50083

Issue Date

2024-06-01

Receive Date

2024-11-01

Publish Date

2024-06-01

Print ISSN

1687-8515

Online ISSN

2682-3578

Link

https://asc.journals.ekb.eg/article_406983.html

Detail API

http://journals.ekb.eg?_action=service&article_code=406983

Order

406,983

Type

Original Article

Type Code

1,549

Publication Type

Journal

Publication Title

Journal of the ACS Advances in Computer Science

Publication Link

https://asc.journals.ekb.eg/

MainTitle

Virtual Keyboard-Mouse in Real-Time using Hand Gesture and Voice Assistant

Details

Type

Article

Created At

01 Feb 2025