Virtual piano

• Main Author: Soh, Dominic Shi Quan
• Other Authors: Ravi Suppiah
• Format: Final Year Project
• Language: English
• Published: 2017
• Subjects: DRNTU::Engineering::Computer science and engineering
• Online Access: http://hdl.handle.net/10356/70393
• Institution: Nanyang Technological University

With the rapid advancements in technology, Smartphones of all brands start to boom around the world. From simple functions like makingcalls and sending text messages, to more complexed functions such as accessing the Internet and taking photos and videos, Smartphones have eventually found its importance in almost everyone’s lives from the moment it was born. With this boom in Smartphones, mobile applications have become the bread and butter of these Smartphones due to the ease of accessibility to these applications via online mobile stores. The possibilities and variations of Smartphone applications are limitlessand are constantly on a rising trend. In order to incorporate more internal functions of a Smartphone, many applications have started to explore integrating with the Smartphone’s camera. This rise in interest in computer vision brings about the need to improvereal-time image processing for Smartphones. This project develops an Android mobile application that uses real-time image processing techniques to provide a close simulationof playing the piano on a piece of paper with piano keys drawn or printed on it. The mobile device is placed on an elevated platform and at an angle for optimum view of the paper piano. When the application is started, the user switches on the piano and fingertip detection, fix the detected keys, and can then start playing on the paper piano. The application tracks fingertips for finger presses on the prefixed piano keys and then plays the corresponding sounds. The application is developed using OpenCV, a library for realtime image processing with over 2500 optimized algorithms. The application is tested on a Samsung Galaxy Note 4 under a single light condition in a small room. The empirical results showed that the proposed set of methodologies worked well for single fingertip keystrokes but failed partially for multiple fingertip keystrokes and more research is needed to implement a robust algorithm to accurately detect multiple keystrokes.