DisMo: A Morphosyntactic, Disfluency and Multi-Word Unit Annotator. An Evaluation on a Corpus of French Spontaneous and Read Speech
International audience We present DisMo, a multi-level annotator for spoken language corpora that integrates part-of-speech tagging with basic disfluency detection and annotation, and multi-word unit recognition. DisMo is a hybrid system that uses a combination of lexical resources, rules, and stati...
Main Authors: | , , |
---|---|
Other Authors: | , , , |
Format: | Conference Object |
Language: | English |
Published: |
HAL CCSD
2014
|
Subjects: | |
Online Access: | https://hal.archives-ouvertes.fr/hal-01703495 https://hal.archives-ouvertes.fr/hal-01703495/document https://hal.archives-ouvertes.fr/hal-01703495/file/2014%20Christodoulides%20et%20al%20-%20DisMo.pdf |
id |
ftunivnantes:oai:HAL:hal-01703495v1 |
---|---|
record_format |
openpolar |
spelling |
ftunivnantes:oai:HAL:hal-01703495v1 2023-05-15T16:50:46+02:00 DisMo: A Morphosyntactic, Disfluency and Multi-Word Unit Annotator. An Evaluation on a Corpus of French Spontaneous and Read Speech Christodoulides, George Avanzi, Mathieu Goldman, Jean-Philippe Institut Langage et Communication (ILC) Université Catholique de Louvain = Catholic University of Louvain (UCL) Université de Neuchâtel (UNINE) Université de Genève = University of Geneva (UNIGE) Reykjavik, Iceland 2014-05-26 https://hal.archives-ouvertes.fr/hal-01703495 https://hal.archives-ouvertes.fr/hal-01703495/document https://hal.archives-ouvertes.fr/hal-01703495/file/2014%20Christodoulides%20et%20al%20-%20DisMo.pdf en eng HAL CCSD info:eu-repo/semantics/altIdentifier/arxiv/1802.02926 hal-01703495 https://hal.archives-ouvertes.fr/hal-01703495 https://hal.archives-ouvertes.fr/hal-01703495/document https://hal.archives-ouvertes.fr/hal-01703495/file/2014%20Christodoulides%20et%20al%20-%20DisMo.pdf ARXIV: 1802.02926 info:eu-repo/semantics/OpenAccess Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC) https://hal.archives-ouvertes.fr/hal-01703495 Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC), May 2014, Reykjavik, Iceland part-of-speech tagging French spoken corpora disfluencies [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] [SCCO.LING]Cognitive science/Linguistics info:eu-repo/semantics/conferenceObject Conference papers 2014 ftunivnantes 2022-10-05T00:45:53Z International audience We present DisMo, a multi-level annotator for spoken language corpora that integrates part-of-speech tagging with basic disfluency detection and annotation, and multi-word unit recognition. DisMo is a hybrid system that uses a combination of lexical resources, rules, and statistical models based on Conditional Random Fields (CRF). In this paper, we present the first public version of DisMo for French. The system is trained and its performance evaluated on a 57k-token corpus, including different varieties of French spoken in three countries (Belgium, France and Switzerland). DisMo supports a multi-level annotation scheme, in which the tokenisation to minimal word units is complemented with multi-word unit groupings (each having associated POS tags), as well as separate levels for annotating disfluencies and discourse phenomena. We present the system's architecture, linguistic resources and its hierarchical tag-set. Results show that DisMo achieves a precision of 95% (finest tag-set) to 96.8% (coarse tag-set) in POS-tagging non-punctuated, sound-aligned transcriptions of spoken French, while also offering substantial possibilities for automated multi-level annotation. Conference Object Iceland Université de Nantes: HAL-UNIV-NANTES |
institution |
Open Polar |
collection |
Université de Nantes: HAL-UNIV-NANTES |
op_collection_id |
ftunivnantes |
language |
English |
topic |
part-of-speech tagging French spoken corpora disfluencies [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] [SCCO.LING]Cognitive science/Linguistics |
spellingShingle |
part-of-speech tagging French spoken corpora disfluencies [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] [SCCO.LING]Cognitive science/Linguistics Christodoulides, George Avanzi, Mathieu Goldman, Jean-Philippe DisMo: A Morphosyntactic, Disfluency and Multi-Word Unit Annotator. An Evaluation on a Corpus of French Spontaneous and Read Speech |
topic_facet |
part-of-speech tagging French spoken corpora disfluencies [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] [SCCO.LING]Cognitive science/Linguistics |
description |
International audience We present DisMo, a multi-level annotator for spoken language corpora that integrates part-of-speech tagging with basic disfluency detection and annotation, and multi-word unit recognition. DisMo is a hybrid system that uses a combination of lexical resources, rules, and statistical models based on Conditional Random Fields (CRF). In this paper, we present the first public version of DisMo for French. The system is trained and its performance evaluated on a 57k-token corpus, including different varieties of French spoken in three countries (Belgium, France and Switzerland). DisMo supports a multi-level annotation scheme, in which the tokenisation to minimal word units is complemented with multi-word unit groupings (each having associated POS tags), as well as separate levels for annotating disfluencies and discourse phenomena. We present the system's architecture, linguistic resources and its hierarchical tag-set. Results show that DisMo achieves a precision of 95% (finest tag-set) to 96.8% (coarse tag-set) in POS-tagging non-punctuated, sound-aligned transcriptions of spoken French, while also offering substantial possibilities for automated multi-level annotation. |
author2 |
Institut Langage et Communication (ILC) Université Catholique de Louvain = Catholic University of Louvain (UCL) Université de Neuchâtel (UNINE) Université de Genève = University of Geneva (UNIGE) |
format |
Conference Object |
author |
Christodoulides, George Avanzi, Mathieu Goldman, Jean-Philippe |
author_facet |
Christodoulides, George Avanzi, Mathieu Goldman, Jean-Philippe |
author_sort |
Christodoulides, George |
title |
DisMo: A Morphosyntactic, Disfluency and Multi-Word Unit Annotator. An Evaluation on a Corpus of French Spontaneous and Read Speech |
title_short |
DisMo: A Morphosyntactic, Disfluency and Multi-Word Unit Annotator. An Evaluation on a Corpus of French Spontaneous and Read Speech |
title_full |
DisMo: A Morphosyntactic, Disfluency and Multi-Word Unit Annotator. An Evaluation on a Corpus of French Spontaneous and Read Speech |
title_fullStr |
DisMo: A Morphosyntactic, Disfluency and Multi-Word Unit Annotator. An Evaluation on a Corpus of French Spontaneous and Read Speech |
title_full_unstemmed |
DisMo: A Morphosyntactic, Disfluency and Multi-Word Unit Annotator. An Evaluation on a Corpus of French Spontaneous and Read Speech |
title_sort |
dismo: a morphosyntactic, disfluency and multi-word unit annotator. an evaluation on a corpus of french spontaneous and read speech |
publisher |
HAL CCSD |
publishDate |
2014 |
url |
https://hal.archives-ouvertes.fr/hal-01703495 https://hal.archives-ouvertes.fr/hal-01703495/document https://hal.archives-ouvertes.fr/hal-01703495/file/2014%20Christodoulides%20et%20al%20-%20DisMo.pdf |
op_coverage |
Reykjavik, Iceland |
genre |
Iceland |
genre_facet |
Iceland |
op_source |
Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC) https://hal.archives-ouvertes.fr/hal-01703495 Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC), May 2014, Reykjavik, Iceland |
op_relation |
info:eu-repo/semantics/altIdentifier/arxiv/1802.02926 hal-01703495 https://hal.archives-ouvertes.fr/hal-01703495 https://hal.archives-ouvertes.fr/hal-01703495/document https://hal.archives-ouvertes.fr/hal-01703495/file/2014%20Christodoulides%20et%20al%20-%20DisMo.pdf ARXIV: 1802.02926 |
op_rights |
info:eu-repo/semantics/OpenAccess |
_version_ |
1766040880708321280 |