Digital Theremin with computer vision

This study presents the methodology and development of a digital Theremin using MediaPipe to enable gesture-based controls for pitch, volume, and vibrato. Through a comprehensive user study, the project evaluated the intuitiveness of the gesture-based controls and the produced sound quality with...

Full description

Saved in:
Bibliographic Details
Main Author: Chua, Ryuichi
Other Authors: Goh Wooi Boon
Format: Final Year Project
Language:English
Published: Nanyang Technological University 2024
Subjects:
Online Access:https://hdl.handle.net/10356/175253
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
Description
Summary:This study presents the methodology and development of a digital Theremin using MediaPipe to enable gesture-based controls for pitch, volume, and vibrato. Through a comprehensive user study, the project evaluated the intuitiveness of the gesture-based controls and the produced sound quality with PyAudio. The digital Theremin was generally well-received, with participants demonstrating high controllability and showing a strong preference for a particular sound version noted for its warmth and expressiveness. Challenges identified include weak detection at screen edges and precise note-playing. Future work includes enhancing responsiveness and gesture recognition accuracy to expand the digital Theremin's appeal. Additionally, future work can investigate leveraging hand gestures in Unity development