Creating percussive rhythms using hand gestures

• Main Author: Han, Si Ding
• Other Authors: Goh Wooi Boon
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computer applications::Computers in other systems; Visual arts and music::Music::Compositions
• Online Access: https://hdl.handle.net/10356/153129
• Institution: Nanyang Technological University

This project explores the creation of interactive music using the LEAP Motion sensor. The LEAP Motion sensor detects hand gestures and movement which helps to investigate how appropriate hand gestures and mid-air interaction can be harnessed to create and control complex percussive rhythms. A part of the investigation involves a comparison between two different hand gestures that are mapped to percussive instruments. A user study was conducted to determine which hand gesture is more intuitive when playing a percussive instrument using the LEAP Motion. Another important investigation involves determining whether the game can effectively teach non-musicians about rhythmic theory, and the degree of ease for musicians to employ their left and right hands to create complex percussive sequences. The system also supports additive layering of different percussive tracks such that complex percussive rendition can be achieved. The user study was conducted to determine the usability of LEAP Motion sensor as a device to play percussive instruments. The sensor provides a different user experience compared to playing the physical percussive instrument due to the lack of touch feedback. Participants of the user study were required to complete tasks which includes experiencing LEAP Motion’s sensory capabilities, playing percussive instrument using two different hand gestures, replicating rhythmic sequences, and creating their own rhythmic patterns.