Cross-speaker viseme mapping using hidden Markov models

• Main Authors: Dong, Liang, Foo, Say Wei, Yong, Lian
• Other Authors: School of Electrical and Electronic Engineering
• Format: Conference or Workshop Item
• Language: English
• Published: 2009
• Subjects: DRNTU::Engineering::Electrical and electronic engineering::Electronic systems
• Online Access: https://hdl.handle.net/10356/91219; http://hdl.handle.net/10220/5953
• Institution: Nanyang Technological University

In this paper, a method of mapping visual speech between different speakers is proposed. This approach adopts Hidden Markov Model (HMM) to model the basic visual speech element – viseme. Some mapping terms are applied to associate the state chains decoded for the visemes produced by different speakers. The HMMs configured in this way are trained using the Baum-Welch estimation, and are used to generate new visemes. Experiments are conducted to map the visemes produced by several speakers to a destination speaker. The experimental results show that the proposed approach provides good accuracy and continuity for mapping the visemes.