Maximum F1-score training for end-to-end mispronunciation detection and diagnosis of L2 English speech

• Main Authors: Yan, Bi-Cheng, Jiang, Shao-Wei Fan, Chao, Fu-An, Chen, Berlin
• Format: Text
• Language: unknown
• Published: 2021
• Subjects: Electrical Engineering and Systems Science - Audio and Speech Processing; Arctic
• Online Access: http://arxiv.org/abs/2108.13816

End-to-end (E2E) neural models are increasingly attracting attention as a promising modeling approach for mispronunciation detection and diagnosis (MDD). Typically, these models are trained by optimizing a cross-entropy criterion, which corresponds to improving the log-likelihood of the training data. However, there is a discrepancy between the objectives of model training and the MDD evaluation, since the performance of an MDD model is commonly evaluated in terms of F1-score instead of phone or word error rate (PER/WER). In view of this, we in this paper explore the use of a discriminative objective function for training E2E MDD models, which aims to maximize the expected F1-score directly. A series of experiments conducted on the L2-ARCTIC dataset show that our proposed method can yield considerable performance improvements in relation to some state-of-the-art E2E MDD approaches and the celebrated GOP method. Comment: Accepted by IEEE International Conference on Multimedia and Expo (ICME 2022)