Virtual piano

• Main Author: Tan, Alan Lai Chian
• Other Authors: Ravi Suppiah
• Format: Final Year Project
• Language: English
• Published: 2016
• Subjects: DRNTU::Engineering
• Online Access: http://hdl.handle.net/10356/66841
• Institution: Nanyang Technological University

Smartphones are small but powerful devices, providing humans with countless uses, from making phone calls and taking photos, to accessing the Internet from almost anywhere. The late twentieth-century saw an explosion of computer applications. Through the installation of apps, the list of possible smartphone uses multiplies by tens of thousands and grows longer every day. With smarter phone cameras coming to the market, there has been great interest in computer vision applications using real-time image processing in mobile devices. This project develops an Android mobile application that uses real-time image processing techniques to provide a close simulation of playing the piano on a piece of paper with piano keys drawn or printed on it. The mobile device is placed on an elevated platform and at an angle for optimum view of the paper piano. When the application is started, the user switches on the piano and fingertip detection, fix the detected keys, and can then start playing on the paper piano. The application tracks fingertips for finger presses on the prefixed piano keys and then plays the corresponding sounds. The application is developed using OpenCV, a library for real-time image processing with over 2500 optimized algorithms. The application is tested on a Samsung Galaxy Note 4 under a single light condition in a small room. The empirical results showed that the proposed set of methodologies worked well for single fingertip keystrokes but failed partially for multiple fingertip keystrokes and more research is needed to implement a robust algorithm to accurately detect multiple keystrokes.