Application of multimodal speech recognition based on deep neural networks in interpretation teaching

• Main Authors: Nai, Ruihua, Hassan, Hanita
• Format: Conference or Workshop Item
• Published: 2023
• Subjects: L Education (General); P Philology. Linguistics
• Online Access: http://eprints.utm.my/108403/; http://dx.doi.org/10.1117/12.3011751
• Institution: Universiti Teknologi Malaysia

In recent years, although speech recognition technology has been widely used, it also faces some problems. This paper studies multimodal speech recognition in interpreting based on deep neural network. Firstly, the deep learning method and its related theoretical basis are introduced. Then, the advantages of speech corpus denoising based on acoustic expert feature extraction and training algorithm, convolution decomposition method and interpretation element analysis are described. Finally, through the experimental verification, it is proved that the recognition system can effectively improve students’ interpretation efficiency and accuracy, and the accuracy rate is more than 93%.