Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects ...
We interact with the world with our hands and see it through our own (egocentric) perspective. A holistic 3Dunderstanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation. Accurately reconstructing such interactions in 3D...
Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , |
---|---|
Format: | Article in Journal/Newspaper |
Language: | unknown |
Published: |
arXiv
2024
|
Subjects: | |
Online Access: | https://dx.doi.org/10.48550/arxiv.2403.16428 https://arxiv.org/abs/2403.16428 |
id |
ftdatacite:10.48550/arxiv.2403.16428 |
---|---|
record_format |
openpolar |
spelling |
ftdatacite:10.48550/arxiv.2403.16428 2024-09-30T14:31:05+00:00 Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects ... Fan, Zicong Ohkawa, Takehiko Yang, Linlin Lin, Nie Zhou, Zhishan Zhou, Shihao Liang, Jiajun Gao, Zhong Zhang, Xuanyang Zhang, Xue Li, Fei Liu, Zheng Lu, Feng Zeid, Karim Abou Leibe, Bastian On, Jeongwan Baek, Seungryul Prakash, Aditya Gupta, Saurabh He, Kun Sato, Yoichi Hilliges, Otmar Chang, Hyung Jin Yao, Angela 2024 https://dx.doi.org/10.48550/arxiv.2403.16428 https://arxiv.org/abs/2403.16428 unknown arXiv arXiv.org perpetual, non-exclusive license http://arxiv.org/licenses/nonexclusive-distrib/1.0/ Computer Vision and Pattern Recognition cs.CV FOS: Computer and information sciences CreativeWork Preprint Article article 2024 ftdatacite https://doi.org/10.48550/arxiv.2403.16428 2024-09-02T08:03:41Z We interact with the world with our hands and see it through our own (egocentric) perspective. A holistic 3Dunderstanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation. Accurately reconstructing such interactions in 3D is challenging due to heavy occlusion, viewpoint bias, camera distortion, and motion blur from the head movement. To this end, we designed the HANDS23 challenge based on the AssemblyHands and ARCTIC datasets with carefully designed training and testing splits. Based on the results of the top submitted methods and more recent baselines on the leaderboards, we perform a thorough analysis on 3D hand(-object) reconstruction tasks. Our analysis demonstrates the effectiveness of addressing distortion specific to egocentric cameras, adopting high-capacity transformers to learn complex hand-object interactions, and fusing predictions from different views. Our study further reveals challenging scenarios intractable with ... : Accepted to ECCV 2024 ... Article in Journal/Newspaper Arctic DataCite Arctic |
institution |
Open Polar |
collection |
DataCite |
op_collection_id |
ftdatacite |
language |
unknown |
topic |
Computer Vision and Pattern Recognition cs.CV FOS: Computer and information sciences |
spellingShingle |
Computer Vision and Pattern Recognition cs.CV FOS: Computer and information sciences Fan, Zicong Ohkawa, Takehiko Yang, Linlin Lin, Nie Zhou, Zhishan Zhou, Shihao Liang, Jiajun Gao, Zhong Zhang, Xuanyang Zhang, Xue Li, Fei Liu, Zheng Lu, Feng Zeid, Karim Abou Leibe, Bastian On, Jeongwan Baek, Seungryul Prakash, Aditya Gupta, Saurabh He, Kun Sato, Yoichi Hilliges, Otmar Chang, Hyung Jin Yao, Angela Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects ... |
topic_facet |
Computer Vision and Pattern Recognition cs.CV FOS: Computer and information sciences |
description |
We interact with the world with our hands and see it through our own (egocentric) perspective. A holistic 3Dunderstanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation. Accurately reconstructing such interactions in 3D is challenging due to heavy occlusion, viewpoint bias, camera distortion, and motion blur from the head movement. To this end, we designed the HANDS23 challenge based on the AssemblyHands and ARCTIC datasets with carefully designed training and testing splits. Based on the results of the top submitted methods and more recent baselines on the leaderboards, we perform a thorough analysis on 3D hand(-object) reconstruction tasks. Our analysis demonstrates the effectiveness of addressing distortion specific to egocentric cameras, adopting high-capacity transformers to learn complex hand-object interactions, and fusing predictions from different views. Our study further reveals challenging scenarios intractable with ... : Accepted to ECCV 2024 ... |
format |
Article in Journal/Newspaper |
author |
Fan, Zicong Ohkawa, Takehiko Yang, Linlin Lin, Nie Zhou, Zhishan Zhou, Shihao Liang, Jiajun Gao, Zhong Zhang, Xuanyang Zhang, Xue Li, Fei Liu, Zheng Lu, Feng Zeid, Karim Abou Leibe, Bastian On, Jeongwan Baek, Seungryul Prakash, Aditya Gupta, Saurabh He, Kun Sato, Yoichi Hilliges, Otmar Chang, Hyung Jin Yao, Angela |
author_facet |
Fan, Zicong Ohkawa, Takehiko Yang, Linlin Lin, Nie Zhou, Zhishan Zhou, Shihao Liang, Jiajun Gao, Zhong Zhang, Xuanyang Zhang, Xue Li, Fei Liu, Zheng Lu, Feng Zeid, Karim Abou Leibe, Bastian On, Jeongwan Baek, Seungryul Prakash, Aditya Gupta, Saurabh He, Kun Sato, Yoichi Hilliges, Otmar Chang, Hyung Jin Yao, Angela |
author_sort |
Fan, Zicong |
title |
Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects ... |
title_short |
Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects ... |
title_full |
Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects ... |
title_fullStr |
Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects ... |
title_full_unstemmed |
Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects ... |
title_sort |
benchmarks and challenges in pose estimation for egocentric hand interactions with objects ... |
publisher |
arXiv |
publishDate |
2024 |
url |
https://dx.doi.org/10.48550/arxiv.2403.16428 https://arxiv.org/abs/2403.16428 |
geographic |
Arctic |
geographic_facet |
Arctic |
genre |
Arctic |
genre_facet |
Arctic |
op_rights |
arXiv.org perpetual, non-exclusive license http://arxiv.org/licenses/nonexclusive-distrib/1.0/ |
op_doi |
https://doi.org/10.48550/arxiv.2403.16428 |
_version_ |
1811635758655602688 |