ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation
Humans intuitively understand that inanimate objects do not move by themselves, but that state changes are typically caused by human manipulation (e.g., the opening of a book). This is not yet the case for machines. In part this is because there exist no datasets with ground-truth 3D annotations for...
Main Authors: | , , , , , , |
---|---|
Format: | Text |
Language: | unknown |
Published: |
2022
|
Subjects: | |
Online Access: | http://arxiv.org/abs/2204.13662 |
id |
ftarxivpreprints:oai:arXiv.org:2204.13662 |
---|---|
record_format |
openpolar |
spelling |
ftarxivpreprints:oai:arXiv.org:2204.13662 2023-09-05T13:16:54+02:00 ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation Fan, Zicong Taheri, Omid Tzionas, Dimitrios Kocabas, Muhammed Kaufmann, Manuel Black, Michael J. Hilliges, Otmar 2022-04-28 http://arxiv.org/abs/2204.13662 unknown http://arxiv.org/abs/2204.13662 Computer Science - Computer Vision and Pattern Recognition text 2022 ftarxivpreprints 2023-08-16T17:03:21Z Humans intuitively understand that inanimate objects do not move by themselves, but that state changes are typically caused by human manipulation (e.g., the opening of a book). This is not yet the case for machines. In part this is because there exist no datasets with ground-truth 3D annotations for the study of physically consistent and synchronised motion of hands and articulated objects. To this end, we introduce ARCTIC -- a dataset of two hands that dexterously manipulate objects, containing 2.1M video frames paired with accurate 3D hand and object meshes and detailed, dynamic contact information. It contains bi-manual articulation of objects such as scissors or laptops, where hand poses and object states evolve jointly in time. We propose two novel articulated hand-object interaction tasks: (1) Consistent motion reconstruction: Given a monocular video, the goal is to reconstruct two hands and articulated objects in 3D, so that their motions are spatio-temporally consistent. (2) Interaction field estimation: Dense relative hand-object distances must be estimated from images. We introduce two baselines ArcticNet and InterField, respectively and evaluate them qualitatively and quantitatively on ARCTIC. Our code and data are available at https://arctic.is.tue.mpg.de. Comment: Project page: https://arctic.is.tue.mpg.de Text Arctic ArcticNet ArXiv.org (Cornell University Library) Arctic |
institution |
Open Polar |
collection |
ArXiv.org (Cornell University Library) |
op_collection_id |
ftarxivpreprints |
language |
unknown |
topic |
Computer Science - Computer Vision and Pattern Recognition |
spellingShingle |
Computer Science - Computer Vision and Pattern Recognition Fan, Zicong Taheri, Omid Tzionas, Dimitrios Kocabas, Muhammed Kaufmann, Manuel Black, Michael J. Hilliges, Otmar ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation |
topic_facet |
Computer Science - Computer Vision and Pattern Recognition |
description |
Humans intuitively understand that inanimate objects do not move by themselves, but that state changes are typically caused by human manipulation (e.g., the opening of a book). This is not yet the case for machines. In part this is because there exist no datasets with ground-truth 3D annotations for the study of physically consistent and synchronised motion of hands and articulated objects. To this end, we introduce ARCTIC -- a dataset of two hands that dexterously manipulate objects, containing 2.1M video frames paired with accurate 3D hand and object meshes and detailed, dynamic contact information. It contains bi-manual articulation of objects such as scissors or laptops, where hand poses and object states evolve jointly in time. We propose two novel articulated hand-object interaction tasks: (1) Consistent motion reconstruction: Given a monocular video, the goal is to reconstruct two hands and articulated objects in 3D, so that their motions are spatio-temporally consistent. (2) Interaction field estimation: Dense relative hand-object distances must be estimated from images. We introduce two baselines ArcticNet and InterField, respectively and evaluate them qualitatively and quantitatively on ARCTIC. Our code and data are available at https://arctic.is.tue.mpg.de. Comment: Project page: https://arctic.is.tue.mpg.de |
format |
Text |
author |
Fan, Zicong Taheri, Omid Tzionas, Dimitrios Kocabas, Muhammed Kaufmann, Manuel Black, Michael J. Hilliges, Otmar |
author_facet |
Fan, Zicong Taheri, Omid Tzionas, Dimitrios Kocabas, Muhammed Kaufmann, Manuel Black, Michael J. Hilliges, Otmar |
author_sort |
Fan, Zicong |
title |
ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation |
title_short |
ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation |
title_full |
ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation |
title_fullStr |
ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation |
title_full_unstemmed |
ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation |
title_sort |
arctic: a dataset for dexterous bimanual hand-object manipulation |
publishDate |
2022 |
url |
http://arxiv.org/abs/2204.13662 |
geographic |
Arctic |
geographic_facet |
Arctic |
genre |
Arctic ArcticNet |
genre_facet |
Arctic ArcticNet |
op_relation |
http://arxiv.org/abs/2204.13662 |
_version_ |
1776198317339836416 |