Non-verbal information estimation in multi-party human-robot/virtual human interaction - Zhang, Zhijie (Nanyang Technological University, 2023)
Robots and virtual agents have been deployed in various fields, and are playing an increasingly important role in human being’s daily lives. Thus, these intelligent agents (IAs) are required to interact with their users appropriately. To achieve this goal, IAs need to understand human social signals, before generating socially acceptable responses. However, current multi-party social human-robot interaction (HRI) is still far from being satisfactory. Unlike dyadic HRI, multi-party HRI involves more than one participant in the interaction, so with the increase in the participant number, IAs face more challenging tasks. The overall objective of this research is to investigate and develop new techniques to empower robots or virtual agents with the ability to understand the behaviors, intentions, and affects of the participants in multi-party social interaction, which helps the agent manage multi-party issues in social HRI. Specifically, this thesis presents new methods to analyze and estimate four types of non-verbal social information in multi-party human-robot interaction scenarios, namely (i) engagement intention estimation, (ii) engagement estimation during interaction, (iii) personality estimation, and (iv) emotion recognition.