Get a Grip: Reconstructing Hand-Object Stable Grasps in Egocentric Videos ...

• Main Authors: Zhu, Zhifan, Damen, Dima
• Format: Article in Journal/Newspaper
• Language: unknown
• Published: arXiv 2023
• Subjects: Computer Vision and Pattern Recognition cs.CV; FOS Computer and information sciences; Arctic
• Online Access: https://dx.doi.org/10.48550/arxiv.2312.15719; https://arxiv.org/abs/2312.15719

We propose the task of Hand-Object Stable Grasp Reconstruction (HO-SGR), the reconstruction of frames during which the hand is stably holding the object. We first develop the stable grasp definition based on the intuition that the in-contact area between the hand and object should remain stable. By analysing the 3D ARCTIC dataset, we identify stable grasp durations and showcase that objects in stable grasps move within a single degree of freedom (1-DoF). We thereby propose a method to jointly optimise all frames within a stable grasp, minimising object motions to a latent 1-DoF. Finally, we extend the knowledge to in-the-wild videos by labelling 2.4K clips of stable grasps. Our proposed EPIC-Grasps dataset includes 390 object instances of 9 categories, featuring stable grasps from videos of daily interactions in 141 environments. Without 3D ground truth, we use stable contact areas and 2D projection masks to assess the HO-SGR task in the wild. We evaluate relevant methods and our approach preserves ... : webpage: https://zhifanzhu.github.io/getagrip ...