ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation

Humans intuitively understand that inanimate objects do not move by themselves, but that state changes are typically caused by human manipulation (e.g., the opening of a book). This is not yet the case for machines. In part this is because there exist no datasets with ground-truth 3D annotations for...

Full description

Bibliographic Details
Main Authors: Fan, Zicong, Taheri, Omid, Tzionas, Dimitrios, Kocabas, Muhammed, Kaufmann, Manuel, Black, Michael J., Hilliges, Otmar
Format: Text
Language:unknown
Published: 2022
Subjects:
Online Access:http://arxiv.org/abs/2204.13662
id ftarxivpreprints:oai:arXiv.org:2204.13662
record_format openpolar
spelling ftarxivpreprints:oai:arXiv.org:2204.13662 2023-09-05T13:16:54+02:00 ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation Fan, Zicong Taheri, Omid Tzionas, Dimitrios Kocabas, Muhammed Kaufmann, Manuel Black, Michael J. Hilliges, Otmar 2022-04-28 http://arxiv.org/abs/2204.13662 unknown http://arxiv.org/abs/2204.13662 Computer Science - Computer Vision and Pattern Recognition text 2022 ftarxivpreprints 2023-08-16T17:03:21Z Humans intuitively understand that inanimate objects do not move by themselves, but that state changes are typically caused by human manipulation (e.g., the opening of a book). This is not yet the case for machines. In part this is because there exist no datasets with ground-truth 3D annotations for the study of physically consistent and synchronised motion of hands and articulated objects. To this end, we introduce ARCTIC -- a dataset of two hands that dexterously manipulate objects, containing 2.1M video frames paired with accurate 3D hand and object meshes and detailed, dynamic contact information. It contains bi-manual articulation of objects such as scissors or laptops, where hand poses and object states evolve jointly in time. We propose two novel articulated hand-object interaction tasks: (1) Consistent motion reconstruction: Given a monocular video, the goal is to reconstruct two hands and articulated objects in 3D, so that their motions are spatio-temporally consistent. (2) Interaction field estimation: Dense relative hand-object distances must be estimated from images. We introduce two baselines ArcticNet and InterField, respectively and evaluate them qualitatively and quantitatively on ARCTIC. Our code and data are available at https://arctic.is.tue.mpg.de. Comment: Project page: https://arctic.is.tue.mpg.de Text Arctic ArcticNet ArXiv.org (Cornell University Library) Arctic
institution Open Polar
collection ArXiv.org (Cornell University Library)
op_collection_id ftarxivpreprints
language unknown
topic Computer Science - Computer Vision and Pattern Recognition
spellingShingle Computer Science - Computer Vision and Pattern Recognition
Fan, Zicong
Taheri, Omid
Tzionas, Dimitrios
Kocabas, Muhammed
Kaufmann, Manuel
Black, Michael J.
Hilliges, Otmar
ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation
topic_facet Computer Science - Computer Vision and Pattern Recognition
description Humans intuitively understand that inanimate objects do not move by themselves, but that state changes are typically caused by human manipulation (e.g., the opening of a book). This is not yet the case for machines. In part this is because there exist no datasets with ground-truth 3D annotations for the study of physically consistent and synchronised motion of hands and articulated objects. To this end, we introduce ARCTIC -- a dataset of two hands that dexterously manipulate objects, containing 2.1M video frames paired with accurate 3D hand and object meshes and detailed, dynamic contact information. It contains bi-manual articulation of objects such as scissors or laptops, where hand poses and object states evolve jointly in time. We propose two novel articulated hand-object interaction tasks: (1) Consistent motion reconstruction: Given a monocular video, the goal is to reconstruct two hands and articulated objects in 3D, so that their motions are spatio-temporally consistent. (2) Interaction field estimation: Dense relative hand-object distances must be estimated from images. We introduce two baselines ArcticNet and InterField, respectively and evaluate them qualitatively and quantitatively on ARCTIC. Our code and data are available at https://arctic.is.tue.mpg.de. Comment: Project page: https://arctic.is.tue.mpg.de
format Text
author Fan, Zicong
Taheri, Omid
Tzionas, Dimitrios
Kocabas, Muhammed
Kaufmann, Manuel
Black, Michael J.
Hilliges, Otmar
author_facet Fan, Zicong
Taheri, Omid
Tzionas, Dimitrios
Kocabas, Muhammed
Kaufmann, Manuel
Black, Michael J.
Hilliges, Otmar
author_sort Fan, Zicong
title ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation
title_short ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation
title_full ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation
title_fullStr ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation
title_full_unstemmed ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation
title_sort arctic: a dataset for dexterous bimanual hand-object manipulation
publishDate 2022
url http://arxiv.org/abs/2204.13662
geographic Arctic
geographic_facet Arctic
genre Arctic
ArcticNet
genre_facet Arctic
ArcticNet
op_relation http://arxiv.org/abs/2204.13662
_version_ 1776198317339836416