Digital Theremin with computer vision - Chua, Ryuichi (Nanyang Technological University, 2024)
This study presents the methodology and development of a digital Theremin using MediaPipe to enable gesture-based controls for pitch, volume, and vibrato. Through a comprehensive user study, the project evaluated the intuitiveness of the gesture-based controls and the produced sound quality with PyAudio. The digital Theremin was generally well-received, with participants demonstrating high controllability and showing a strong preference for a particular sound version noted for its warmth and expressiveness. Challenges identified include weak detection at screen edges and precise note-playing. Future work includes enhancing responsiveness and gesture recognition accuracy to expand the digital Theremin's appeal. Additionally, future work can investigate leveraging hand gestures in Unity development