Interactive gesture-based music on a mobile phone

• Main Author: Ng, Joseph Heng Qi
• Other Authors: Goh Wooi Boon
• Format: Final Year Project
• Language: English
• Published: 2019
• Subjects: DRNTU::Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision; DRNTU::Visual arts and music::Performing arts
• Online Access: http://hdl.handle.net/10356/77759
• Institution: Nanyang Technological University

This project is concerned with using hand gestures to produce musical sounds, using a mobile phone and its camera functionality. As such, the end goal involves the user being able to, with the aid of additional props, generate musical sounds out of their mobile phones by simply using hand gestures related movements. It is important to note that the purpose of the project is not to create a musical instrument that can easily replace other traditional instruments such as the guitar or the piano, but merely serve as a new form of musical expression tool, that is not explored deeply before. The end user should have very basic ideas of how music theory works in order to use the application fully, but there is no technical skill required beforehand, thus all users would be able to use the application without difficulty. The project would utilize the Android platform, with the target API level of 21. The OpenCV library is used for image processing, and the JSyn library is used for audio synthesis.