Transformer-based joint learning approach for text normalization in Vietnamese Automatic Speech Recognition Systems

• Main Authors: BUI, The Viet, LUONG, Tho Chi, TRAN, Oanh Thi
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: ASR; named entity recognition; post-processing; punctuator; text normalization; transformer-based joint learning models; Numerical Analysis and Scientific Computing; South and Southeast Asian Languages and Societies; Theory and Algorithms
• Online Access: https://ink.library.smu.edu.sg/sis_research/7591; https://ink.library.smu.edu.sg/context/sis_research/article/8594/viewcontent/TransformerBasedVietnameseASR_av.pdf
• Institution: Singapore Management University

In this article, we investigate the task of normalizing transcribed texts in Vietnamese Automatic Speech Recognition (ASR) systems in order to improve user readability and the performance of downstream tasks. This task usually consists of two main sub-tasks: predicting and inserting punctuation (i.e., period, comma); and detecting and standardizing named entities (i.e., numbers, person names) from spoken forms to their appropriate written forms. To achieve these goals, we introduce a complete corpus including of 87,700 sentences and investigate conditional joint learning approaches which globally optimize two sub-tasks simultaneously. The experimental results are quite promising. Overall, the proposed architecture outperformed the conventional architecture which trains individual models on the two sub-tasks separately. The joint models are furthered improved when integrated with the surrounding contexts (SCs). Specifically, we obtained 81.13% for the first sub-task and 94.41% for the second sub-task in the F1 scores using the best model.