ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation ...

• Main Authors: Fan, Zicong, Taheri, Omid, Tzionas, Dimitrios, Kocabas, Muhammed, Kaufmann, Manuel, Black, Michael J., Hilliges, Otmar
• Format: Text
• Language: English
• Published: ETH Zurich 2023
• Subjects: Arctic
• Online Access: https://dx.doi.org/10.3929/ethz-b-000642263; http://hdl.handle.net/20.500.11850/642263

Humans intuitively understand that inanimate objects do not move by themselves, but that state changes are typically caused by human manipulation (e.g., the opening of a book). This is not yet the case for machines. In part this is because there exist no datasets with ground-truth 3D annotations for the study of physically consistent and synchronised motion of hands and articulated objects. To this end, we introduce ARCTIC - a dataset of two hands that dexterously manipulate objects, containing 2.1M video frames paired with accurate 3D hand and object meshes and detailed, dynamic contact information. It contains bi-manual articulation of objects such as scissors or laptops, where hand poses and object states evolve jointly in time. We propose two novel articulated hand-object interaction tasks: (1) Consistent motion reconstruction: Given a monocular video, the goal is to reconstruct two hands and articulated objects in 3D, so that their motions are spatio-temporally consistent. (2) Interaction field ... : ISSN:1063-6919 ...