Learning from the master: Distilling cross-modal advanced knowledge for lip reading

• Main Authors: REN, Sucheng, DU, Yong, LV, Jianming, HAN, Guoqiang, HE, Shengfeng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2021
• Subjects: Curricula; Modal analysis; Speech recognition; Audio signal; Collaborative framework; Cross-modal; Interactive strategy; Learning designs; Lip reading; Modal data; Performance; Teachers'; Teaching contents; Students; Databases and Information Systems; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/8442; https://ink.library.smu.edu.sg/context/sis_research/article/9445/viewcontent/Ren_Learning_From_the_Master_Distilling_Cross_Modal_Advanced_Knowledge_for_Lip_CVPR_2021_paper.pdf
• Institution: Singapore Management University

Lip reading aims to predict the spoken sentences from silent lip videos. Due to the fact that such a vision task usually performs worse than its counterpart speech recognition, one potential scheme is to distill knowledge from a teacher pretrained by audio signals. However, the latent domain gap bet...