DLVS4Audio2Sheet: Deep learning-based vocal separation for audio into music sheet conversion

• Main Authors: TEO, Nicole, WANG, Zhaoxia, GHE, Ezekiel, TAN, Yee Sen, OKTAVIO, Kevan, LEWI, Alexander Vincent, ZHANG, Allyne, HO, Seng-Beng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Music; Choral audio; Music sheet; Vocal separation; Audio-to-Sheet; Deep learning; Open-Unmix; Band-Split Recurrent Neural Networks (BSRNN); Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/9160; https://ink.library.smu.edu.sg/context/sis_research/article/10163/viewcontent/4._DLVS4Audio2Sheet_Deep_Learning_based_Vocal_Separation_for_Audio_into_Music_Sheet_Conversion.pdf
• Institution: Singapore Management University

While manual transcription tools exist, music enthusiasts, including amateur singers, still encounter challenges when transcribing performances into sheet music. This paper addresses the complex task of translating music audio into music sheets, particularly challenging in the intricate field of choral arrangements where multiple voices intertwine. We propose DLVS4Audio2Sheet, a novel method leveraging advanced deep learning models, Open-Unmix and Band-Split Recurrent Neural Networks (BSRNN), for vocal separation. DLVS4Audio2Sheet segments choral audio into individual vocal sections and selects the optimal model for further processing, aiming towards audio into music sheet conversion. We evaluate DLVS4Audio2Sheet’s performance using these deep learning algorithms and assess its effectiveness in producing isolated vocals suitable for notated scoring music conversion. By ensuring superior vocal separation quality through model selection, DLVS4Audio2Sheet enhances audio into music sheet conversion. This research contributes to the advancement of music technology by thoroughly exploring state-of-the-art models, methodologies, and techniques for converting choral audio into music sheets. Code and datasets are available at: https://github.com/DevGoliath/DLVS4Audio2Sheet.