Demonstrating multi-modal human instruction comprehension with AR smart glass

Demonstrating multi-modal human instruction comprehension with AR smart glass

We present a multi-modal human instruction comprehension prototype for object acquisition tasks that involve verbal, visual and pointing gesture cues. Our prototype includes an AR smart-glass for issuing the instructions and a Jetson TX2 pervasive device for executing comprehension algorithms. With...

Full description

Saved in:
Bibliographic Details
Main Authors: WEERAKOON, Mudiyanselage Dulanga Kaveesha, SUBBARAJU, Vigneshwaran, TRAN, Tuan, MISRA, Archan
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2023
Subjects:
Human-AI Collaboration
Multi-Modal Networks
Pervasive Systems
Referring Expression Comprehension
Visual Grounding
Software Engineering
Online Access:https://ink.library.smu.edu.sg/sis_research/7797
https://ink.library.smu.edu.sg/context/sis_research/article/8800/viewcontent/COMSNETS2023_Demo_CR.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
Description
Summary:We present a multi-modal human instruction comprehension prototype for object acquisition tasks that involve verbal, visual and pointing gesture cues. Our prototype includes an AR smart-glass for issuing the instructions and a Jetson TX2 pervasive device for executing comprehension algorithms. With this setup, we enable on-device, computationally efficient object acquisition task comprehension with an average latency in the range of 150-330msec.

Similar Items