Demonstrating multi-modal human instruction comprehension with AR smart glass

Demonstrating multi-modal human instruction comprehension with AR smart glass

We present a multi-modal human instruction comprehension prototype for object acquisition tasks that involve verbal, visual and pointing gesture cues. Our prototype includes an AR smart-glass for issuing the instructions and a Jetson TX2 pervasive device for executing comprehension algorithms. With...

Full description

Saved in:
Bibliographic Details
Main Authors: WEERAKOON, Mudiyanselage Dulanga Kaveesha, SUBBARAJU, Vigneshwaran, TRAN, Tuan, MISRA, Archan
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2023
Subjects:
Human-AI Collaboration
Multi-Modal Networks
Pervasive Systems
Referring Expression Comprehension
Visual Grounding
Software Engineering
Online Access:https://ink.library.smu.edu.sg/sis_research/7797
https://ink.library.smu.edu.sg/context/sis_research/article/8800/viewcontent/COMSNETS2023_Demo_CR.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-8800
record_format dspace
spelling sg-smu-ink.sis_research-88002023-04-04T03:12:37Z Demonstrating multi-modal human instruction comprehension with AR smart glass WEERAKOON, Mudiyanselage Dulanga Kaveesha SUBBARAJU, Vigneshwaran TRAN, Tuan MISRA, Archan We present a multi-modal human instruction comprehension prototype for object acquisition tasks that involve verbal, visual and pointing gesture cues. Our prototype includes an AR smart-glass for issuing the instructions and a Jetson TX2 pervasive device for executing comprehension algorithms. With this setup, we enable on-device, computationally efficient object acquisition task comprehension with an average latency in the range of 150-330msec. 2023-01-01T08:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/7797 info:doi/10.1109/COMSNETS56262.2023.10041269 https://ink.library.smu.edu.sg/context/sis_research/article/8800/viewcontent/COMSNETS2023_Demo_CR.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Human-AI Collaboration Multi-Modal Networks Pervasive Systems Referring Expression Comprehension Visual Grounding Software Engineering
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Human-AI Collaboration
Multi-Modal Networks
Pervasive Systems
Referring Expression Comprehension
Visual Grounding
Software Engineering
spellingShingle Human-AI Collaboration
Multi-Modal Networks
Pervasive Systems
Referring Expression Comprehension
Visual Grounding
Software Engineering
WEERAKOON, Mudiyanselage Dulanga Kaveesha
SUBBARAJU, Vigneshwaran
TRAN, Tuan
MISRA, Archan
Demonstrating multi-modal human instruction comprehension with AR smart glass
description We present a multi-modal human instruction comprehension prototype for object acquisition tasks that involve verbal, visual and pointing gesture cues. Our prototype includes an AR smart-glass for issuing the instructions and a Jetson TX2 pervasive device for executing comprehension algorithms. With this setup, we enable on-device, computationally efficient object acquisition task comprehension with an average latency in the range of 150-330msec.
format text
author WEERAKOON, Mudiyanselage Dulanga Kaveesha
SUBBARAJU, Vigneshwaran
TRAN, Tuan
MISRA, Archan
author_facet WEERAKOON, Mudiyanselage Dulanga Kaveesha
SUBBARAJU, Vigneshwaran
TRAN, Tuan
MISRA, Archan
author_sort WEERAKOON, Mudiyanselage Dulanga Kaveesha
title Demonstrating multi-modal human instruction comprehension with AR smart glass
title_short Demonstrating multi-modal human instruction comprehension with AR smart glass
title_full Demonstrating multi-modal human instruction comprehension with AR smart glass
title_fullStr Demonstrating multi-modal human instruction comprehension with AR smart glass
title_full_unstemmed Demonstrating multi-modal human instruction comprehension with AR smart glass
title_sort demonstrating multi-modal human instruction comprehension with ar smart glass
publisher Institutional Knowledge at Singapore Management University
publishDate 2023
url https://ink.library.smu.edu.sg/sis_research/7797
https://ink.library.smu.edu.sg/context/sis_research/article/8800/viewcontent/COMSNETS2023_Demo_CR.pdf
_version_ 1770576515522625536

Similar Items