Mining a Multimodal Corpus for Non-Verbal Signals Sequences Conveying Attitudes

• Main Authors: Chollet, Mathieu, Ochs, Magalie, Pelachaud, Catherine
• Other Authors: Laboratoire Traitement et Communication de l'Information (LTCI), Télécom ParisTech-Institut Mines-Télécom Paris (IMT)-Centre National de la Recherche Scientifique (CNRS)
• Format: Conference Object
• Language: English
• Published: HAL CCSD 2014
• Subjects: Non-verbal behavior; Interpersonal attitudes; Sequence mining; [INFO.INFO-HC]Computer Science [cs]/Human-Computer Interaction [cs.HC]; Iceland
• Online Access: https://hal.science/hal-01074879; https://hal.science/hal-01074879/document; https://hal.science/hal-01074879/file/lreccamerareadyfinal.pdf

International audience Interpersonal attitudes are expressed by non-verbal behaviors on a variety of different modalities. The perception of these behaviors is influenced by how they are sequenced with other behaviors from the same person and behaviors from other interactants. In this paper, we present a method for extracting and generating sequences of non-verbal signals expressing interpersonal attitudes. These sequences are used as part of a framework for non-verbal expression with Embodied Conversational Agents that considers different features of non-verbal behavior: global behavior tendencies, interpersonal reactions, sequencing of non-verbal signals, and communicative intentions. Our method uses a sequence mining technique on an annotated multimodal corpus to extract sequences characteristic of different attitudes. New sequences of non-verbal signals are generated using a probabilistic model, and evaluated using the previously mined sequences.