Voice conversion with parallel/non-parallel data and synthetic speech detection

• Main Author: Tian, Xiaohai
• Other Authors: Chng Eng Siong
• Format: Theses and Dissertations
• Language: English
• Published: 2019
• Subjects: DRNTU::Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/89880; http://hdl.handle.net/10220/47729
• Institution: Nanyang Technological University

The objective of voice conversion techniques is to convert a source speaker's voice so that it sounds like that of a target speaker. Voice conversion belongs to a popular area of personalized speech generation. On one hand, it can be applied to solve problems, e.g. emotion conversion, improving the intelligibility of speech, or change whisper/murmur into speech. On the other hand, voice conversion also presents a threat to Automatic Speaker Verification (ASV) systems. Synthetic speech detection is a technique to discriminate between live and synthetic speech. In this sense, it provides a feasible way to improve robustness and protect ASV systems. This thesis focuses on the following two aspects: improving performance in voice conversion and studying countermeasures for synthetic speech detection. For voice conversion, a typical system usually requires parallel data, which means that the source and target utterances contain the same linguistic content for conversion model training. As parallel data is always difficult to collect in practice, the challenge is to estimate a robust conversion function with limited parallel data or even non-parallel data. In this thesis, we propose two novel voice conversion methods to improve the system performance for both parallel and non-parallel data based voice conversion. First, we propose a novel parallel data based voice conversion framework, in which we combine frequency warping and exemplar-based methods. Under an exemplar-based frequency warping (EFW) framework, each warping function and spectral residual is generated from dictionaries of warping function exemplars and spectral residual exemplars respectively. With sparsity constraint, we avoid statistical averaging effects due to Gaussian Mixture Models (GMM) and provide a more accurate warping function and residual compensation. Results from experiments on VOICES database show that the proposed method significantly improved speech quality compared with state-of-the-art parametric methods. Second, we propose a non-parallel data based voice conversion framework that we call the Average Modeling Approach (AMA). The proposed approach makes use of a multi-speaker average model that maps speaker independent linguistic features to speaker dependent acoustic features. As linguistic features and acoustic features can be extracted from the same utterance, the proposed approach doesn't require parallel data for average model training and adaptation. Two average model adaptation approaches are introduced, namely model-based adaptation and feature-based adaptation. Experiments on the Voice Conversion Challenge 2018 (VCC2018) database confirmed the effectiveness of the proposed method. For synthetic speech detection, we investigated the use of different feature representations to discriminate between live and synthetic speech. As low-dimensional features are typically used in synthetic techniques, detailed information has been abandoned in synthetic speech. This work mainly focused on the detailed information of high-dimensional features, especially their high-frequency parts, for synthetic speech detection. Dynamic features were also calculated to assess the effectiveness of temporal information to detect artifacts across frames. Results of experiments on the standard ASVspoof 2015 corpus suggest that high-dimensional features and dynamic features are useful for synthetic speech detection.