SocioPhone: Everyday Face-to-Face Interaction Monitoring Platform Using Multi-Phone Sensor Fusion

• Main Authors: LEE, Youngki, MIN, Chulhong, HWANG, Chanyou, LEE, Jaeung, HWANG, Inseok, JU, Younghyun, YOO, Chungkuk, MOON, Miri, LEE, Uichin, SONG, Junehwa
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2013
• Subjects: Conversation; Interaction; Platform; Social; Volume Topography; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/2064; https://ink.library.smu.edu.sg/context/sis_research/article/3063/viewcontent/p375_lee.pdf
• Institution: Singapore Management University

In this paper, we propose SocioPhone, a novel initiative to build a mobile platform for face-to-face interaction monitoring. Face-to-face interaction, especially conversation, is a fundamental part of everyday life. Interaction-aware applications aimed at facilitating group conversations have been proposed, but have not proliferated yet. Useful contexts to capture and support face-to-face interactions need to be explored more deeply. More important, recognizing delicate conversational contexts with commodity mobile devices requires solving a number of technical challenges. As a first step to address such challenges, we identify useful meta-linguistic contexts of conversation, such as turn-takings, prosodic features, a dominant participant, and pace. These serve as cornerstones for building a variety of interaction-aware applications. SocioPhone abstracts such useful meta-linguistic contexts as a set of intuitive APIs. Its runtime efficiently monitors registered contexts during in-progress conversations and notifies applications on-the-fly. Importantly, we have noticed that online turn monitoring is the basic building block for extracting diverse meta-linguistic contexts, and have devised a novel volume-topography-based method. We show the usefulness of SocioPhone with several interesting applications: SocioTherapist, SocioDigest, and Tug-of-War. Also, we show that our turn-monitoring technique is highly accurate and energy-efficient under diverse real-life situations.