Mispronunciation Detection and Correction via Discrete Acoustic Units

Computer-Assisted Pronunciation Training (CAPT) plays an important role in language learning. However, conventional CAPT methods cannot effectively use non-native utterances for supervised training because the ground truth pronunciation needs expensive annotation. Meanwhile, certain undefined nonnat...

Full description

Bibliographic Details
Main Authors: Zhang, Zhan, Wang, Yuehai, Yang, Jianyi
Format: Article in Journal/Newspaper
Language:unknown
Published: arXiv 2021
Subjects:
Audio and Speech Processing eess.AS
Sound cs.SD
FOS Electrical engineering, electronic engineering, information engineering
FOS Computer and information sciences
Arctic
Vae
Online Access:https://dx.doi.org/10.48550/arxiv.2108.05517
https://arxiv.org/abs/2108.05517
Description
Summary:Computer-Assisted Pronunciation Training (CAPT) plays an important role in language learning. However, conventional CAPT methods cannot effectively use non-native utterances for supervised training because the ground truth pronunciation needs expensive annotation. Meanwhile, certain undefined nonnative phonemes cannot be correctly classified into standard phonemes. To solve these problems, we use the vector-quantized variational autoencoder (VQ-VAE) to encode the speech into discrete acoustic units in a self-supervised manner. Based on these units, we propose a novel method that integrates both discriminative and generative models. The proposed method can detect mispronunciation and generate the correct pronunciation at the same time. Experiments on the L2-Arctic dataset show that the detection F1 score is improved by 9.58% relatively compared with recognition-based methods. The proposed method also achieves a comparable word error rate (WER) and the best style preservation for mispronunciation correction compared with text-to-speech (TTS) methods. : 5 pages, 4 figures, (IEEE SPL under review)

Similar Items