3D Hand Pose Estimation in Egocentric Images in the Wild
We present WildHands, a method for 3D hand pose estimation in egocentric images in the wild. This is challenging due to (a) lack of 3D hand pose annotations for images in the wild, and (b) a form of perspective distortion-induced shape ambiguity that arises in the analysis of crops around hands. For...
Main Authors: | , , , |
---|---|
Format: | Text |
Language: | unknown |
Published: |
2023
|
Subjects: | |
Online Access: | http://arxiv.org/abs/2312.06583 |
id |
ftarxivpreprints:oai:arXiv.org:2312.06583 |
---|---|
record_format |
openpolar |
spelling |
ftarxivpreprints:oai:arXiv.org:2312.06583 2024-01-14T10:04:41+01:00 3D Hand Pose Estimation in Egocentric Images in the Wild Prakash, Aditya Tu, Ruisen Chang, Matthew Gupta, Saurabh 2023-12-11 http://arxiv.org/abs/2312.06583 unknown http://arxiv.org/abs/2312.06583 Computer Science - Computer Vision and Pattern Recognition Computer Science - Artificial Intelligence Computer Science - Machine Learning Computer Science - Robotics text 2023 ftarxivpreprints 2023-12-17T02:07:43Z We present WildHands, a method for 3D hand pose estimation in egocentric images in the wild. This is challenging due to (a) lack of 3D hand pose annotations for images in the wild, and (b) a form of perspective distortion-induced shape ambiguity that arises in the analysis of crops around hands. For the former, we use auxiliary supervision on in-the-wild data in the form of segmentation masks & grasp labels in addition to 3D supervision available in lab datasets. For the latter, we provide spatial cues about the location of the hand crop in the camera's field of view. Our approach achieves the best 3D hand pose on the ARCTIC leaderboard and outperforms FrankMocap, a popular and robust approach for estimating hand pose in the wild, by 45.3% when evaluated on 2D hand pose on our EPIC-HandKps dataset. Comment: Project page: https://ap229997.github.io/projects/hands/ Text Arctic ArXiv.org (Cornell University Library) Arctic |
institution |
Open Polar |
collection |
ArXiv.org (Cornell University Library) |
op_collection_id |
ftarxivpreprints |
language |
unknown |
topic |
Computer Science - Computer Vision and Pattern Recognition Computer Science - Artificial Intelligence Computer Science - Machine Learning Computer Science - Robotics |
spellingShingle |
Computer Science - Computer Vision and Pattern Recognition Computer Science - Artificial Intelligence Computer Science - Machine Learning Computer Science - Robotics Prakash, Aditya Tu, Ruisen Chang, Matthew Gupta, Saurabh 3D Hand Pose Estimation in Egocentric Images in the Wild |
topic_facet |
Computer Science - Computer Vision and Pattern Recognition Computer Science - Artificial Intelligence Computer Science - Machine Learning Computer Science - Robotics |
description |
We present WildHands, a method for 3D hand pose estimation in egocentric images in the wild. This is challenging due to (a) lack of 3D hand pose annotations for images in the wild, and (b) a form of perspective distortion-induced shape ambiguity that arises in the analysis of crops around hands. For the former, we use auxiliary supervision on in-the-wild data in the form of segmentation masks & grasp labels in addition to 3D supervision available in lab datasets. For the latter, we provide spatial cues about the location of the hand crop in the camera's field of view. Our approach achieves the best 3D hand pose on the ARCTIC leaderboard and outperforms FrankMocap, a popular and robust approach for estimating hand pose in the wild, by 45.3% when evaluated on 2D hand pose on our EPIC-HandKps dataset. Comment: Project page: https://ap229997.github.io/projects/hands/ |
format |
Text |
author |
Prakash, Aditya Tu, Ruisen Chang, Matthew Gupta, Saurabh |
author_facet |
Prakash, Aditya Tu, Ruisen Chang, Matthew Gupta, Saurabh |
author_sort |
Prakash, Aditya |
title |
3D Hand Pose Estimation in Egocentric Images in the Wild |
title_short |
3D Hand Pose Estimation in Egocentric Images in the Wild |
title_full |
3D Hand Pose Estimation in Egocentric Images in the Wild |
title_fullStr |
3D Hand Pose Estimation in Egocentric Images in the Wild |
title_full_unstemmed |
3D Hand Pose Estimation in Egocentric Images in the Wild |
title_sort |
3d hand pose estimation in egocentric images in the wild |
publishDate |
2023 |
url |
http://arxiv.org/abs/2312.06583 |
geographic |
Arctic |
geographic_facet |
Arctic |
genre |
Arctic |
genre_facet |
Arctic |
op_relation |
http://arxiv.org/abs/2312.06583 |
_version_ |
1788059162573275136 |