JollyGesture: Exploring dual-purpose gestures in VR presentations

• Main Authors: PARK, Gun Woo Warren, TANG, Anthony, CHEVALIER, Fanny
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Gestural input; Virtual Reality; Presentation; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/9165; https://ink.library.smu.edu.sg/context/sis_research/article/10168/viewcontent/33_jollygesture_exploring_dual_pu.pdf
• Institution: Singapore Management University

Virtual reality (VR) offers new opportunities for presenters to use expressive body language to engage their audience. Yet, most VR presentation systems have adopted control mechanisms that mimic those found in face-to-face presentation systems. We explore the use of gestures that have dual-purpose: first, for the audience, a communicative purpose; second, for the presenter, a control purpose to alter content in slides. To support presenters, we provide guidance on what gestures are available and their effects. We realize our design approach in JollyGesture, a VR technology probe that recognizes dual-purpose gestures in a presentation scenario. We evaluate our approach through a design study with 12 participants, where in addition to using JollyGesture to deliver a mock presentation, we asked them to imagine gestures with the same communicative and control purpose, before and after being exposed to our probe. The study revealed several new design avenues valuable for VR presentation system design: expressive and coarse-grained communicative gestures, as well as subtle and hidden gestures intended for system control. Our work suggests that VR presentation systems of the future that embrace expressive body language will face design tensions relating to task loading and authenticity.