ClearSpeech: Improving voice quality of earbuds using both in-ear and out-ear microphones

• Main Authors: MA, Dong, DANG, Ting, DING, Ming, BALAN, Rajesh Krishna
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Speech enhancement; Smart earbuds; Earables; Audio processing; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/8579; https://ink.library.smu.edu.sg/context/sis_research/article/9582/viewcontent/ClearSpeech.pdf
• Institution: Singapore Management University

Wireless earbuds have been gaining increasing popularity and using them to make phone calls or issue voice commands requires the earbud microphones to pick up human speech. When the speaker is in a noisy environment, speech quality degrades significantly and requires speech enhancement (SE). In this paper, we present ClearSpeech, a novel deep-learningbased SE system designed for wireless earbuds. Specifically, by jointly using the earbud’s in-ear and out-ear microphones, we devised a suite of techniques to effectively fuse the two signals and enhance the magnitude and phase of the speech spectrogram. We built an earbud prototype to evaluate ClearSpeech under various settings with data collected from 20 subjects. Our results suggest that ClearSpeech can improve the SE performance significantly compared to conventional approaches using the out-ear microphone only. We also show that ClearSpeech can process user speech in real-time on smartphones.