Bubbleu: Exploring augmented reality game design with uncertain AI-based interaction

• Main Authors: KIM, Minji, LEE, Kyungjin, BALAN, Rajesh Krishna, LEE, Youngki
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: computer vision; vision sensing; Human-AI Interaction; Artificial Intelligence and Robotics; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/8652; https://ink.library.smu.edu.sg/context/sis_research/article/9655/viewcontent/3544548.3581270.pdf
• Institution: Singapore Management University

Object detection, while being an attractive interaction method for Augmented Reality (AR), is fundamentally error-prone due to the probabilistic nature of the underlying AI models, resulting in sub-optimal user experiences. In this paper, we explore the effect of three game design concepts, Ambiguity, Transparency, and Controllability, to provide better gameplay experiences in AR games that use error-prone object detection-based interaction modalities. First, we developed a base AR pet breeding game, called Bubbleu that uses object detection as a key interaction method. We then implemented three different variants, each according to the three concepts, to investigate the impact of each design concept on the overall user experience. Our user study results show that each design has its own strengths and can improve player experiences in different ways such as decreasing perceived errors (Ambiguity), explaining the system (Transparency), and enabling users to control the rate of uncertainties (Controllability).