Implementation of an Automatic Sign Language Lexical Annotation Framework based on Propositional Dynamic Logic

• Main Authors: Curiel Diaz, Arturo Tlacaélel, Collet, Christophe
• Other Authors: Traitement et Compréhension d’Images (IRIT-TCI), Institut de recherche en informatique de Toulouse (IRIT), Université Toulouse Capitole (UT Capitole), Université de Toulouse (UT)-Université de Toulouse (UT)-Université Toulouse - Jean Jaurès (UT2J), Université de Toulouse (UT)-Université Toulouse III - Paul Sabatier (UT3), Université de Toulouse (UT)-Centre National de la Recherche Scientifique (CNRS)-Institut National Polytechnique (Toulouse) (Toulouse INP), Université de Toulouse (UT)-Toulouse Mind & Brain Institut (TMBI), Université Toulouse - Jean Jaurès (UT2J), Université de Toulouse (UT)-Université de Toulouse (UT)-Université Toulouse III - Paul Sabatier (UT3), Université de Toulouse (UT)-Université Toulouse Capitole (UT Capitole), Université de Toulouse (UT), Université Toulouse III - Paul Sabatier (UT3)
• Format: Conference Object
• Language: English
• Published: HAL CCSD 2014
• Subjects: Sign language framework; Automatic annotation; Propositional dynamic logic; [INFO.INFO-GR]Computer Science [cs]/Graphics [cs.GR]; [INFO.INFO-TS]Computer Science [cs]/Signal and Image Processing; [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV]; [INFO.INFO-CV]Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV]; [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]; Iceland
• Online Access: https://hal.science/hal-01135588; https://hal.science/hal-01135588/document; https://hal.science/hal-01135588/file/Curiel_13113.pdf

International audience In this paper, we present the implementation of an automatic Sign Language (SL) sign annotation framework based on a formal logic, the Propositional Dynamic Logic (PDL). Our system relies heavily on the use of a specific variant of PDL, the Propositional Dynamic Logic for Sign Language (PDLSL), which lets us describe SL signs as formulae and corpora videos as labeled transition systems (LTSs). Here, we intend to show how a generic annotation system can be constructed upon these underlying theoretical principles, regardless of the tracking technologies available or the input format of corpora. With this in mind, we generated a development framework that adapts the system to specific use cases. Furthermore, we present some results obtained by our application when adapted to one distinct case, 2D corpora analysis with pre-processed tracking information. We also present some insights on how such a technology can be used to analyze 3D real-time data, captured with a depth device.