A MIXED REALITY TECHNOLOGY IMPLEMENTATION WITH AUDIO SIGNAL PROCESSING TO TRAIN AND BOOST MOTIVATION ON LEARNING KEYBOARD INSTRUMENT

• Main Author: Ganang Putranda, Insan
• Format: Theses
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/76683
• Institution: Institut Teknologi Bandung

Technology becomes increasingly important in education, including music, through innovative immersive applications that can help in learning and practicing music. A popular musical instruments used for practice is keyboard instruments, such as piano. Learning piano is not easy and requires a long time, leading to many students dropping out. Various VR (virtual reality) and AR (augmented reality) applications have been developed to improve skills and knowledge in playing musical instruments. However, these applications mainly utilize MIDI (Musical Instrument Digital Interface) input for validating notes. This research explores the potential implementation of mixed reality, which is a part of AR, through sound processing capabilites or without using the MIDI, and game approach to enhance learning and motivation. The research used the HoloLens 2 device and the FFT (Fast Fourier Transform) method with peak detection and compared various window functions to determine the most accurate one. Factors influencing motivation were analyzed using the HMSAM (Hedonic-Motivation System Adoption Model). The results showed that Blackman-Harris window function performed the best, with a 97.28% accuracy rate when tested on complex songs. Moreover, out of the 31 users tested and evaluated using HMSAM, it was revealed that curiosity and joy were the most significant factors influencing the use of the application. The effectiveness level of the learning was moderate, with an increase of 31.28%. Although there were limitations in the use of audio processing, it could still be utilized and improved further to keep users motivated to learn piano.