Multi-way Variational NMT for UGC: Improving Robustness in Zero-shot Scenarios via Mixture Density Networks

• Main Authors: Rosales Núñez, José, Carlos, Seddah, Djamé, Wisniewski, Guillaume
• Other Authors: Laboratoire Interdisciplinaire des Sciences du Numérique (LISN), Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS), Automatic Language Modelling and ANAlysis & Computational Humanities (ALMAnaCH), Inria de Paris, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria), Laboratoire de Linguistique Formelle (LLF - UMR7110), Centre National de la Recherche Scientifique (CNRS)-Université Paris Cité (UPCité), ANR-16-CE33-0021,PARSITI,Analyser l'impossible, Traduire l'improbable(2016)
• Format: Conference Object
• Language: English
• Published: HAL CCSD 2023
• Subjects: Machine Translation; User-generated content; [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG]; Faroe Islands; Torshavn
• Online Access: https://hal.science/hal-04384748; https://hal.science/hal-04384748/document; https://hal.science/hal-04384748/file/_Article__VNMT_nodalida.pdf

International audience This work presents a novel Variational Neural Machine Translation (VNMT) architecture with enhanced robustness properties, which we investigate through a detailed case-study addressing noisy French user-generated content (UGC) translation to English. We show that the proposed model, with results comparable or superior to state-of-the-art VNMT, improves performance over UGC translation in a zero-shot evaluation scenario while keeping optimal translation scores on in-domain test sets. We elaborate on such results by visualizing and explaining how neural learning representations behave when processing UGC noise. In addition, we show that VNMT enforces robustness to the learned embeddings, which can be later used for robust transfer learning approaches.