Gesture recognition with transparent solar cells

• Main Authors: MA, Dong, LAN, Guohao, HASSAN, Mahbub, HU, Wen, UPAMA, B. Mushfika, UDDIN, Ashraf, YOUSEEF, Moustafa
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2018
• Subjects: Gesture Recognition; Transparent Solar Cell; Solar Energy Harvesting; Artificial Intelligence and Robotics; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/7003; https://ink.library.smu.edu.sg/context/sis_research/article/8006/viewcontent/3267204.3267209.pdf
• Institution: Singapore Management University

Transparent solar cell is an emerging solar energy harvesting technology that allows us to see through these cells. This revolutionary discovery is creating unique opportunities to turn any mobile device screen into solar energy harvester. In this paper, we consider the possibility of using such energy harvesting screens as a sensor to detect hand gestures. As different gestures impact the incident light on the screen in a different way, they are expected to create unique energy generation patterns for the transparent solar cell. Our goal is to recognize gestures by detecting these solar energy patterns. A key uncertainty we face with transparent solar cell is that, to provide transparency, they cannot harvest from the visible spectra, which may lead to weaker energy patterns for the gestures. To study gesture recognition feasibility of transparent solar cell, we develop a 1cmx1cm organic see-through solar cell which provides high level of content visibility when placed on mobile phone screen. We then use the output current of the organic cell as the source signal for gesture pattern recognition using machine learning. Experimental results demonstrate that we can detect five hand gestures with average accuracies of 95%. We also compare gesture recognition accuracies of our prototype organic cell with those obtained from a conventional ceramic opaque solar cell, which reveals that organic solar cell can recognize some of these gestures almost as good as the opaque cells.