Converting vocal performances into sheet music leveraging large language models

• Main Authors: JIANG, Jinjing, NICOLE ANNE HUI-YING TEO, PEN, Haibo, HO, Seng-Beng, WANG, Zhaoxia
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: natural language processing; vocal transcription; sheet music automation; large language models; a cappella; music analysis; vocal melodies; music composition; singing voice simulation; transcription tools; Artificial Intelligence and Robotics
• Online Access: https://ink.library.smu.edu.sg/sis_research/9704; https://ink.library.smu.edu.sg/context/sis_research/article/10704/viewcontent/sentire2024teo.pdf
• Institution: Singapore Management University

Advanced natural language processing (NLP) models are increasingly applied in music composition and performance, particularly for generating vocal melodies and simulating singing voices. While NLP techniques have been effective in analyzing vocal performance data to assess quality and style, the automatic transcription of vocal performances into sheet music remains a significant challenge. Manual transcription tools often fall short due to the intricate dynamics of vocal expression. This study tackles the automation of vocal performance transcription into sheet music using innovative techniques, including large language models (LLMs). We propose a method to translate vocal audio input into display-ready sheet music effectively. Our findings reveal the strengths and limitations of various approaches, especially in transcribing a cappella performances into notes and lyrics. This research advances the field of NLP-driven music analysis and underscores the transformative potential of these models in vocal transcription.