Test Set Sampling Affects System Rankings: Expanded Human Evaluation of WMT20 English-Inuktitut Systems ...
We present a collection of expanded human annotations of the WMT20 English-Inuktitut machine translation shared task, covering the Nunavut Hansard portion of the dataset. Additionally, we recompute News rankings to take into account the completed set of human annotations and certain irregularities i...
Main Author: | |
---|---|
Format: | Article in Journal/Newspaper |
Language: | unknown |
Published: |
Underline Science Inc.
2022
|
Subjects: | |
Online Access: | https://dx.doi.org/10.48448/qjpp-1k02 https://underline.io/lecture/67166-test-set-sampling-affects-system-rankings-expanded-human-evaluation-of-wmt20-english-inuktitut-systems |
id |
ftdatacite:10.48448/qjpp-1k02 |
---|---|
record_format |
openpolar |
spelling |
ftdatacite:10.48448/qjpp-1k02 2023-10-01T03:57:02+02:00 Test Set Sampling Affects System Rankings: Expanded Human Evaluation of WMT20 English-Inuktitut Systems ... Association for Computational Linguistics 2022 2022 https://dx.doi.org/10.48448/qjpp-1k02 https://underline.io/lecture/67166-test-set-sampling-affects-system-rankings-expanded-human-evaluation-of-wmt20-english-inuktitut-systems unknown Underline Science Inc. article MediaObject Conference talk Audiovisual 2022 ftdatacite https://doi.org/10.48448/qjpp-1k02 2023-09-04T13:27:46Z We present a collection of expanded human annotations of the WMT20 English-Inuktitut machine translation shared task, covering the Nunavut Hansard portion of the dataset. Additionally, we recompute News rankings to take into account the completed set of human annotations and certain irregularities in the annotation task construction. We show the effect of these changes on the downstream task of the evaluation of automatic metrics. Finally, we demonstrate that character-level metrics correlate well with human judgments for the task of automatically evaluating translation into this polysynthetic language. ... Article in Journal/Newspaper inuktitut Nunavut DataCite Metadata Store (German National Library of Science and Technology) Nunavut |
institution |
Open Polar |
collection |
DataCite Metadata Store (German National Library of Science and Technology) |
op_collection_id |
ftdatacite |
language |
unknown |
description |
We present a collection of expanded human annotations of the WMT20 English-Inuktitut machine translation shared task, covering the Nunavut Hansard portion of the dataset. Additionally, we recompute News rankings to take into account the completed set of human annotations and certain irregularities in the annotation task construction. We show the effect of these changes on the downstream task of the evaluation of automatic metrics. Finally, we demonstrate that character-level metrics correlate well with human judgments for the task of automatically evaluating translation into this polysynthetic language. ... |
format |
Article in Journal/Newspaper |
author |
Association for Computational Linguistics 2022 |
spellingShingle |
Association for Computational Linguistics 2022 Test Set Sampling Affects System Rankings: Expanded Human Evaluation of WMT20 English-Inuktitut Systems ... |
author_facet |
Association for Computational Linguistics 2022 |
author_sort |
Association for Computational Linguistics 2022 |
title |
Test Set Sampling Affects System Rankings: Expanded Human Evaluation of WMT20 English-Inuktitut Systems ... |
title_short |
Test Set Sampling Affects System Rankings: Expanded Human Evaluation of WMT20 English-Inuktitut Systems ... |
title_full |
Test Set Sampling Affects System Rankings: Expanded Human Evaluation of WMT20 English-Inuktitut Systems ... |
title_fullStr |
Test Set Sampling Affects System Rankings: Expanded Human Evaluation of WMT20 English-Inuktitut Systems ... |
title_full_unstemmed |
Test Set Sampling Affects System Rankings: Expanded Human Evaluation of WMT20 English-Inuktitut Systems ... |
title_sort |
test set sampling affects system rankings: expanded human evaluation of wmt20 english-inuktitut systems ... |
publisher |
Underline Science Inc. |
publishDate |
2022 |
url |
https://dx.doi.org/10.48448/qjpp-1k02 https://underline.io/lecture/67166-test-set-sampling-affects-system-rankings-expanded-human-evaluation-of-wmt20-english-inuktitut-systems |
geographic |
Nunavut |
geographic_facet |
Nunavut |
genre |
inuktitut Nunavut |
genre_facet |
inuktitut Nunavut |
op_doi |
https://doi.org/10.48448/qjpp-1k02 |
_version_ |
1778527934351409152 |