mmHand: 3D hand pose estimation using millimeter-wave radar

An Dong; Dalong Zhang; Yong Huang; Chaoran Su

doi:10.1117/12.3034122

13 June 2024 mmHand: 3D hand pose estimation using millimeter-wave radar

An Dong, Dalong Zhang, Yong Huang, Chaoran Su

Proceedings Volume 13180, International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2024); 131805Y (2024) https://doi.org/10.1117/12.3034122
Event: International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2024), 2024, Guangzhou, China

Abstract

Hand pose recognition provides attractive applications, including smart homes, AR/VR, etc, which can facilitate HumanComputer Interaction to solve problems. Traditional vision-based hand pose recognition fail in poor lighting, and also have privacy issues, while mmWave signals can effectively solve these problems, while protecting privacy. In this paper, we propose mmHand, a system for high-precision and fine-grained gesture recognition using commercial millimeter-wave devices. Specifically, we use millimeter-wave radar to collect data, generate point clouds of the environment, and use camera for cross-modal supervised training to process and enhance the point clouds. Finally, cross-attention mechanism is used to achieve recognition of hand joints from the enhanced point clouds data. Extensive experimental results show that the MPJPE (Mean Per Joint Position Error) of mmHand is 0.45cm, and when the normalization error is 0.2, the recognition accuracy can reach 94%. The MPJPE in different environments and distances are both at low level, which demonstrates the superior robustness and effectiveness of our system.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

An Dong, Dalong Zhang, Yong Huang, and Chaoran Su "mmHand: 3D hand pose estimation using millimeter-wave radar", Proc. SPIE 13180, International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2024), 131805Y (13 June 2024); https://doi.org/10.1117/12.3034122

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Point clouds

Gesture recognition

Radar

Cameras

Human computer interaction

Reflection

Education and training

Show All Keywords

Keywords/Phrases

Search In:

Publication Years