ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation
Humans intuitively understand that inanimate objects do not move by themselves, but that state changes are typically caused by human manipulation (e.g., the opening of a book). This is not yet the case for machines. In part this is because there exist no datasets with ground-truth 3D annotations for...
Main Authors: | , , , , , , , , |
---|---|
Format: | Conference Object |
Language: | English |
Published: |
IEEE
2023
|
Subjects: | |
Online Access: | https://hdl.handle.net/20.500.11850/642263 https://doi.org/10.3929/ethz-b-000642263 |
id |
ftethz:oai:www.research-collection.ethz.ch:20.500.11850/642263 |
---|---|
record_format |
openpolar |
spelling |
ftethz:oai:www.research-collection.ethz.ch:20.500.11850/642263 2024-02-04T09:57:36+01:00 ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation Fan, Zicong Taheri, Omid Tzionas, Dimitrios Kocabas, Muhammed Kaufmann, Manuel id_orcid:0 000-0001-5309-319X Black, Michael J. Hilliges, Otmar id_orcid:0 000-0002-5068-3474 2023 application/application/pdf https://hdl.handle.net/20.500.11850/642263 https://doi.org/10.3929/ethz-b-000642263 en eng IEEE info:eu-repo/semantics/altIdentifier/doi/10.1109/CVPR52729.2023.01244 info:eu-repo/semantics/altIdentifier/isbn/979-8-3503-0129-8 info:eu-repo/semantics/altIdentifier/wos/001062522105025 http://hdl.handle.net/20.500.11850/642263 doi:10.3929/ethz-b-000642263 urn:isbn:979-8-3503-0129-8 info:eu-repo/semantics/openAccess http://rightsstatements.org/page/InC-NC/1.0/ In Copyright - Non-Commercial Use Permitted 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) info:eu-repo/semantics/conferenceObject Conference Paper info:eu-repo/semantics/acceptedVersion 2023 ftethz https://doi.org/20.500.11850/64226310.3929/ethz-b-00064226310.1109/CVPR52729.2023.01244 2024-01-08T00:53:01Z Humans intuitively understand that inanimate objects do not move by themselves, but that state changes are typically caused by human manipulation (e.g., the opening of a book). This is not yet the case for machines. In part this is because there exist no datasets with ground-truth 3D annotations for the study of physically consistent and synchronised motion of hands and articulated objects. To this end, we introduce ARCTIC - a dataset of two hands that dexterously manipulate objects, containing 2.1M video frames paired with accurate 3D hand and object meshes and detailed, dynamic contact information. It contains bi-manual articulation of objects such as scissors or laptops, where hand poses and object states evolve jointly in time. We propose two novel articulated hand-object interaction tasks: (1) Consistent motion reconstruction: Given a monocular video, the goal is to reconstruct two hands and articulated objects in 3D, so that their motions are spatio-temporally consistent. (2) Interaction field estimation: Dense relative hand-object distances must be estimated from images. We introduce two baselines ArcticNet and InterField, respectively and evaluate them qualitatively and quantitatively on ARCTIC. Our code and data are available at https://arctic.is.tue.mpg.de. ISSN:1063-6919 Conference Object Arctic ArcticNet ETH Zürich Research Collection Arctic |
institution |
Open Polar |
collection |
ETH Zürich Research Collection |
op_collection_id |
ftethz |
language |
English |
description |
Humans intuitively understand that inanimate objects do not move by themselves, but that state changes are typically caused by human manipulation (e.g., the opening of a book). This is not yet the case for machines. In part this is because there exist no datasets with ground-truth 3D annotations for the study of physically consistent and synchronised motion of hands and articulated objects. To this end, we introduce ARCTIC - a dataset of two hands that dexterously manipulate objects, containing 2.1M video frames paired with accurate 3D hand and object meshes and detailed, dynamic contact information. It contains bi-manual articulation of objects such as scissors or laptops, where hand poses and object states evolve jointly in time. We propose two novel articulated hand-object interaction tasks: (1) Consistent motion reconstruction: Given a monocular video, the goal is to reconstruct two hands and articulated objects in 3D, so that their motions are spatio-temporally consistent. (2) Interaction field estimation: Dense relative hand-object distances must be estimated from images. We introduce two baselines ArcticNet and InterField, respectively and evaluate them qualitatively and quantitatively on ARCTIC. Our code and data are available at https://arctic.is.tue.mpg.de. ISSN:1063-6919 |
format |
Conference Object |
author |
Fan, Zicong Taheri, Omid Tzionas, Dimitrios Kocabas, Muhammed Kaufmann, Manuel id_orcid:0 000-0001-5309-319X Black, Michael J. Hilliges, Otmar id_orcid:0 000-0002-5068-3474 |
spellingShingle |
Fan, Zicong Taheri, Omid Tzionas, Dimitrios Kocabas, Muhammed Kaufmann, Manuel id_orcid:0 000-0001-5309-319X Black, Michael J. Hilliges, Otmar id_orcid:0 000-0002-5068-3474 ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation |
author_facet |
Fan, Zicong Taheri, Omid Tzionas, Dimitrios Kocabas, Muhammed Kaufmann, Manuel id_orcid:0 000-0001-5309-319X Black, Michael J. Hilliges, Otmar id_orcid:0 000-0002-5068-3474 |
author_sort |
Fan, Zicong |
title |
ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation |
title_short |
ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation |
title_full |
ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation |
title_fullStr |
ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation |
title_full_unstemmed |
ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation |
title_sort |
arctic: a dataset for dexterous bimanual hand-object manipulation |
publisher |
IEEE |
publishDate |
2023 |
url |
https://hdl.handle.net/20.500.11850/642263 https://doi.org/10.3929/ethz-b-000642263 |
geographic |
Arctic |
geographic_facet |
Arctic |
genre |
Arctic ArcticNet |
genre_facet |
Arctic ArcticNet |
op_source |
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |
op_relation |
info:eu-repo/semantics/altIdentifier/doi/10.1109/CVPR52729.2023.01244 info:eu-repo/semantics/altIdentifier/isbn/979-8-3503-0129-8 info:eu-repo/semantics/altIdentifier/wos/001062522105025 http://hdl.handle.net/20.500.11850/642263 doi:10.3929/ethz-b-000642263 urn:isbn:979-8-3503-0129-8 |
op_rights |
info:eu-repo/semantics/openAccess http://rightsstatements.org/page/InC-NC/1.0/ In Copyright - Non-Commercial Use Permitted |
op_doi |
https://doi.org/20.500.11850/64226310.3929/ethz-b-00064226310.1109/CVPR52729.2023.01244 |
_version_ |
1789961922511110144 |