Computer-based tracking, analysis, and visualization of linguistically significant nonmanual events in American Sign Language (ASL)
Our linguistically annotated American Sign Language (ASL) corpora have formed a basis for research to automate detection by computer of essential linguistic information conveyed through facial expressions and head movements. We have tracked head position and facial deformations, and used computation...
Main Authors: | , , , , , |
---|---|
Other Authors: | , , , , , , , , |
Format: | Other/Unknown Material |
Language: | English |
Published: |
EUROPEAN LANGUAGE RESOURCES ASSOC-ELRA
2014
|
Subjects: | |
Online Access: | https://hdl.handle.net/2144/31880 http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000355611001071&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=6e74115fe3da270499c3d65c9b17d654 |
id |
ftbostonuniv:oai:open.bu.edu:2144/31880 |
---|---|
record_format |
openpolar |
spelling |
ftbostonuniv:oai:open.bu.edu:2144/31880 2023-05-15T16:50:52+02:00 Computer-based tracking, analysis, and visualization of linguistically significant nonmanual events in American Sign Language (ASL) Neidle, Carol Liu, Jingjing Liu, Bo Peng, Xi Vogler, Christian Metaxas, Dimitris Calzolari, N. Choukri, K. Declerck, T. Loftsson, H. Maegaard, B. Mariani, J. Moreno, A. Odijk, J. Piperidis, S. Reykjavik, ICELAND 2014-01-01 8 p. https://hdl.handle.net/2144/31880 http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000355611001071&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=6e74115fe3da270499c3d65c9b17d654 English eng EUROPEAN LANGUAGE RESOURCES ASSOC-ELRA LREC 2014 - NINTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000355611001071&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=6e74115fe3da270499c3d65c9b17d654 Carol Neidle, Jingjing Liu, Bo Liu, Xi Peng, Christian Vogler, Dimitris Metaxas. 2014. "Computer-based Tracking, Analysis, and Visualization of Linguistically Significant Nonmanual Events in American Sign Language (ASL)." LREC 2014 - NINTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION. 9th International Conference on Language Resources and Evaluation (LREC). Reykjavik, ICELAND, 2014-05-26 - 2014-05-31. https://hdl.handle.net/2144/31880 Linguistics American Sign Language (ASL) Social sciences Language & linguistics Nonmanual grammatical marking Computer-based sign language recognition Conference materials 2014 ftbostonuniv 2022-07-11T12:03:14Z Our linguistically annotated American Sign Language (ASL) corpora have formed a basis for research to automate detection by computer of essential linguistic information conveyed through facial expressions and head movements. We have tracked head position and facial deformations, and used computational learning to discern specific grammatical markings. Our ability to detect, identify, and temporally localize the occurrence of such markings in ASL videos has recently been improved by incorporation of (1) new techniques for deformable model-based 3D tracking of head position and facial expressions, which provide significantly better tracking accuracy and recover quickly from temporary loss of track due to occlusion; and (2) a computational learning approach incorporating 2-level Conditional Random Fields (CRFs), suited to the multi-scale spatio-temporal characteristics of the data, which analyses not only low-level appearance characteristics, but also the patterns that enable identification of significant gestural components, such as periodic head movements and raised or lowered eyebrows. Here we summarize our linguistically motivated computational approach and the results for detection and recognition of nonmanual grammatical markings; demonstrate our data visualizations, and discuss the relevance for linguistic research; and describe work underway to enable such visualizations to be produced over large corpora and shared publicly on the Web. Other/Unknown Material Iceland Boston University: OpenBU |
institution |
Open Polar |
collection |
Boston University: OpenBU |
op_collection_id |
ftbostonuniv |
language |
English |
topic |
Linguistics American Sign Language (ASL) Social sciences Language & linguistics Nonmanual grammatical marking Computer-based sign language recognition |
spellingShingle |
Linguistics American Sign Language (ASL) Social sciences Language & linguistics Nonmanual grammatical marking Computer-based sign language recognition Neidle, Carol Liu, Jingjing Liu, Bo Peng, Xi Vogler, Christian Metaxas, Dimitris Computer-based tracking, analysis, and visualization of linguistically significant nonmanual events in American Sign Language (ASL) |
topic_facet |
Linguistics American Sign Language (ASL) Social sciences Language & linguistics Nonmanual grammatical marking Computer-based sign language recognition |
description |
Our linguistically annotated American Sign Language (ASL) corpora have formed a basis for research to automate detection by computer of essential linguistic information conveyed through facial expressions and head movements. We have tracked head position and facial deformations, and used computational learning to discern specific grammatical markings. Our ability to detect, identify, and temporally localize the occurrence of such markings in ASL videos has recently been improved by incorporation of (1) new techniques for deformable model-based 3D tracking of head position and facial expressions, which provide significantly better tracking accuracy and recover quickly from temporary loss of track due to occlusion; and (2) a computational learning approach incorporating 2-level Conditional Random Fields (CRFs), suited to the multi-scale spatio-temporal characteristics of the data, which analyses not only low-level appearance characteristics, but also the patterns that enable identification of significant gestural components, such as periodic head movements and raised or lowered eyebrows. Here we summarize our linguistically motivated computational approach and the results for detection and recognition of nonmanual grammatical markings; demonstrate our data visualizations, and discuss the relevance for linguistic research; and describe work underway to enable such visualizations to be produced over large corpora and shared publicly on the Web. |
author2 |
Calzolari, N. Choukri, K. Declerck, T. Loftsson, H. Maegaard, B. Mariani, J. Moreno, A. Odijk, J. Piperidis, S. |
format |
Other/Unknown Material |
author |
Neidle, Carol Liu, Jingjing Liu, Bo Peng, Xi Vogler, Christian Metaxas, Dimitris |
author_facet |
Neidle, Carol Liu, Jingjing Liu, Bo Peng, Xi Vogler, Christian Metaxas, Dimitris |
author_sort |
Neidle, Carol |
title |
Computer-based tracking, analysis, and visualization of linguistically significant nonmanual events in American Sign Language (ASL) |
title_short |
Computer-based tracking, analysis, and visualization of linguistically significant nonmanual events in American Sign Language (ASL) |
title_full |
Computer-based tracking, analysis, and visualization of linguistically significant nonmanual events in American Sign Language (ASL) |
title_fullStr |
Computer-based tracking, analysis, and visualization of linguistically significant nonmanual events in American Sign Language (ASL) |
title_full_unstemmed |
Computer-based tracking, analysis, and visualization of linguistically significant nonmanual events in American Sign Language (ASL) |
title_sort |
computer-based tracking, analysis, and visualization of linguistically significant nonmanual events in american sign language (asl) |
publisher |
EUROPEAN LANGUAGE RESOURCES ASSOC-ELRA |
publishDate |
2014 |
url |
https://hdl.handle.net/2144/31880 http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000355611001071&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=6e74115fe3da270499c3d65c9b17d654 |
op_coverage |
Reykjavik, ICELAND |
genre |
Iceland |
genre_facet |
Iceland |
op_relation |
LREC 2014 - NINTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000355611001071&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=6e74115fe3da270499c3d65c9b17d654 Carol Neidle, Jingjing Liu, Bo Liu, Xi Peng, Christian Vogler, Dimitris Metaxas. 2014. "Computer-based Tracking, Analysis, and Visualization of Linguistically Significant Nonmanual Events in American Sign Language (ASL)." LREC 2014 - NINTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION. 9th International Conference on Language Resources and Evaluation (LREC). Reykjavik, ICELAND, 2014-05-26 - 2014-05-31. https://hdl.handle.net/2144/31880 |
_version_ |
1766040992969916416 |