Real-time robotic hand control using human gesture recognition

V. Egipko; V. Voronin; N. Gapon; M. Zhdanova; E. Semenishchev; A. Zelensky

doi:10.1117/12.2666559

13 June 2023 Real-time robotic hand control using human gesture recognition

V. Egipko, V. Voronin, N. Gapon, M. Zhdanova, E. Semenishchev, A. Zelensky

Proceedings Volume 12528, Real-Time Image Processing and Deep Learning 2023; 125280E (2023) https://doi.org/10.1117/12.2666559
Event: SPIE Defense + Commercial Sensing, 2023, Orlando, Florida, United States

Conference Poster

Abstract

Currently, there are many options for controlling robotic devices. Human-machine interaction is a key component of the control infrastructure. The most common solution is mobile devices or embedded touch screens, as well as next generation virtual reality devices. In human-machine interaction, most input devices are controlled manually, which is not always convenient, and sometimes even impossible. One option is gesture control, which has become increasingly common in the last few years. This artificial cognitive “sensory perception” or ability is a communication channel between a human and a machine. This article presents a two-steps approach to real-time control robotic devices. The first step is the hand recognition method base on palm detection (SSD Detector) and hand landmark models. After a palm detection, the hand landmark model performs fine localization of the key points of the 3-D coordinate of the hand inside the detected areas of the hand through regression and direct coordinate prediction. The model learns a consistent internal representation of the hand posture and is resistant to even partially visible hands and self-occlusions. The second human gesture recognition step is based on obtaining the coordinates of the hand, the distance from the camera to the hand in space, raised, lowered fingers and other indicators that allow you to accurately determine the shown gesture. In terms of gesture recognition accuracy, the proposed real-time system is better than the state-of-the-art methods.

Citation Download Citation

V. Egipko, V. Voronin, N. Gapon, M. Zhdanova, E. Semenishchev, and A. Zelensky "Real-time robotic hand control using human gesture recognition", Proc. SPIE 12528, Real-Time Image Processing and Deep Learning 2023, 125280E (13 June 2023); https://doi.org/10.1117/12.2666559

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available