Inferring smartphone keypress via smartwatch inertial sensing

• Main Authors: SEN, Sougata, GROVER, Karan, SUBBARAJU, Vigneshwaran, MISRA, Archan
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2017
• Subjects: Inertial navigation systems; Smartphones; Ubiquitous computing; Wearable sensors; White noise; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/3583; https://ink.library.smu.edu.sg/context/sis_research/article/4584/viewcontent/wristSense__1_.pdf
• Institution: Singapore Management University

Due to numerous benefits, sensor-rich smartwatchesand wrist-worn wearable devices are quickly gaining popularity.The popularity of these devices also raises privacy concerns. Inthis paper we explore one such privacy concern: the possibility ofextracting the location of a user’s touch-event on a smartphone,using the inertial sensor data of a smartwatch worn by the useron the same arm. This is a major concern not only because itmight be possible for an attacker to extract private and sensitiveinformation from the inputs provided but also because the attackmode utilises a device (smartwatch) that is distinct from thedevice being attacked (smartphone). Through a user study wefind that such attacks are possible. Specifically, we can infer theuser’s entry pattern on a qwerty keyboard, with an error boundof ±2 neighboring keys, with 73.85% accuracy. As a possiblepreventive mechanism, we also show that adding a little whitenoise to inertial sensor data can reduce the inference accuracyby almost 30%, without affecting the accuracy of macro-gesturerecognition.