Multimodal mobile sensing systems for physiological and psychological assessment

• Main Author: HUYNH, Nguyen Phan Sinh
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2019
• Subjects: Mobile sensing; physiological sensing; Programming Languages and Compilers; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/etd_coll/249; https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=1249&context=etd_coll
• Institution: Singapore Management University

Sensing systems for monitoring physiological and psychological states have been studied extensively in both academic and industry research for different applications across various domains. However, most of the studies have been done in the lab environment with controlled and complicated sensor setup, which is only suitable for serious healthcare applications in which the obtrusiveness and immobility can be compromised in a trade-off for accurate clinical screening or diagnosing. The recent substantial development of mobile devices with embedded miniaturized sensors are now allowing new opportunities to adapt and develop such sensing systems in the mobile context. The ability to sense physiological and psychological state using mobile (and wearable) sensors would make its applications much more feasible and accessible for daily use in different domains such as healthcare, education, security, media and entertainment. Still, there are several research challenges remain in order to develop mobile sensing systems that can monitor users’ physiological signals and psychological conditions accurately and effectively. This thesis will address three key aspects related to realizing the multimodal mobile sensing systems for physiological and psychological state assessment. First, as the mobile embedded sensors are not designed exclusively for physiological sensing purpose, we attempt to improve the sensing capabilities of mobile devices to acquire the vital physiological signals. Specifically, we study the feasibility of using mobile sensors to measure a set of vital physiological signals, in particular, the cardiovascular metrics including blood volume, heartbeat-to-heart beat interval, heart rate, and heart rate variability. The changes in those physiological signals are essential in detecting many psychological states. Second, we validate the importance of assessing the physiological and psychological states in mobile context across various domains. Lastly, we develop and evaluate a multimodal sensing system to measure engagement level of mobile gamers. While the focus of our study was on mobile gaming scenario, we believe the concept of such sensing system is applicable to improve user experience in other mobile activities, including playing games, watching advertisements, or studying using their mobile devices.