Reducing Confusion in Active Learning for Part-Of-Speech Tagging

Active learning (AL) uses a data selection algorithm to select useful training samples to minimize annotation cost. This is now an essential tool for building low-resource syntactic analyzers such as part-of-speech (POS) taggers. Existing AL heuristics are generally designed on the principle of sele...

Full description

Bibliographic Details
Main Authors: Chaudhary, Aditi, Anastasopoulos, Antonios, Sheikh, Zaid, Neubig, Graham
Format: Article in Journal/Newspaper
Language:unknown
Published: arXiv 2020
Subjects:
Computation and Language cs.CL
FOS Computer and information sciences
sami
Online Access:https://dx.doi.org/10.48550/arxiv.2011.00767
https://arxiv.org/abs/2011.00767
id ftdatacite:10.48550/arxiv.2011.00767
record_format openpolar
spelling ftdatacite:10.48550/arxiv.2011.00767 2023-05-15T18:12:36+02:00 Reducing Confusion in Active Learning for Part-Of-Speech Tagging Chaudhary, Aditi Anastasopoulos, Antonios Sheikh, Zaid Neubig, Graham 2020 https://dx.doi.org/10.48550/arxiv.2011.00767 https://arxiv.org/abs/2011.00767 unknown arXiv arXiv.org perpetual, non-exclusive license http://arxiv.org/licenses/nonexclusive-distrib/1.0/ Computation and Language cs.CL FOS Computer and information sciences Article CreativeWork article Preprint 2020 ftdatacite https://doi.org/10.48550/arxiv.2011.00767 2022-03-10T15:11:07Z Active learning (AL) uses a data selection algorithm to select useful training samples to minimize annotation cost. This is now an essential tool for building low-resource syntactic analyzers such as part-of-speech (POS) taggers. Existing AL heuristics are generally designed on the principle of selecting uncertain yet representative training instances, where annotating these instances may reduce a large number of errors. However, in an empirical study across six typologically diverse languages (German, Swedish, Galician, North Sami, Persian, and Ukrainian), we found the surprising result that even in an oracle scenario where we know the true uncertainty of predictions, these current heuristics are far from optimal. Based on this analysis, we pose the problem of AL as selecting instances which maximally reduce the confusion between particular pairs of output tags. Extensive experimentation on the aforementioned languages shows that our proposed AL strategy outperforms other AL strategies by a significant margin. We also present auxiliary results demonstrating the importance of proper calibration of models, which we ensure through cross-view training, and analysis demonstrating how our proposed strategy selects examples that more closely follow the oracle data distribution. : To appear in TACL 2020. This is a pre-MIT Press publication version Article in Journal/Newspaper sami DataCite Metadata Store (German National Library of Science and Technology)
institution Open Polar
collection DataCite Metadata Store (German National Library of Science and Technology)
op_collection_id ftdatacite
language unknown
topic Computation and Language cs.CL
FOS Computer and information sciences
spellingShingle Computation and Language cs.CL
FOS Computer and information sciences
Chaudhary, Aditi
Anastasopoulos, Antonios
Sheikh, Zaid
Neubig, Graham
Reducing Confusion in Active Learning for Part-Of-Speech Tagging
topic_facet Computation and Language cs.CL
FOS Computer and information sciences
description Active learning (AL) uses a data selection algorithm to select useful training samples to minimize annotation cost. This is now an essential tool for building low-resource syntactic analyzers such as part-of-speech (POS) taggers. Existing AL heuristics are generally designed on the principle of selecting uncertain yet representative training instances, where annotating these instances may reduce a large number of errors. However, in an empirical study across six typologically diverse languages (German, Swedish, Galician, North Sami, Persian, and Ukrainian), we found the surprising result that even in an oracle scenario where we know the true uncertainty of predictions, these current heuristics are far from optimal. Based on this analysis, we pose the problem of AL as selecting instances which maximally reduce the confusion between particular pairs of output tags. Extensive experimentation on the aforementioned languages shows that our proposed AL strategy outperforms other AL strategies by a significant margin. We also present auxiliary results demonstrating the importance of proper calibration of models, which we ensure through cross-view training, and analysis demonstrating how our proposed strategy selects examples that more closely follow the oracle data distribution. : To appear in TACL 2020. This is a pre-MIT Press publication version
format Article in Journal/Newspaper
author Chaudhary, Aditi
Anastasopoulos, Antonios
Sheikh, Zaid
Neubig, Graham
author_facet Chaudhary, Aditi
Anastasopoulos, Antonios
Sheikh, Zaid
Neubig, Graham
author_sort Chaudhary, Aditi
title Reducing Confusion in Active Learning for Part-Of-Speech Tagging
title_short Reducing Confusion in Active Learning for Part-Of-Speech Tagging
title_full Reducing Confusion in Active Learning for Part-Of-Speech Tagging
title_fullStr Reducing Confusion in Active Learning for Part-Of-Speech Tagging
title_full_unstemmed Reducing Confusion in Active Learning for Part-Of-Speech Tagging
title_sort reducing confusion in active learning for part-of-speech tagging
publisher arXiv
publishDate 2020
url https://dx.doi.org/10.48550/arxiv.2011.00767
https://arxiv.org/abs/2011.00767
genre sami
genre_facet sami
op_rights arXiv.org perpetual, non-exclusive license
http://arxiv.org/licenses/nonexclusive-distrib/1.0/
op_doi https://doi.org/10.48550/arxiv.2011.00767
_version_ 1766185114681737216

Similar Items