Demo abstract: VGGlass - Demonstrating visual grounding and localization synergy with a LiDAR-enabled smart-glass

Demo abstract: VGGlass - Demonstrating visual grounding and localization synergy with a LiDAR-enabled smart-glass

This work demonstrates the VGGlass system, which simultaneously interprets human instructions for a target acquisition task and determines the precise 3D positions of both user and the target object. This is achieved by utilizing LiDARs mounted in the infrastructure and a smart glass device worn by...

Full description

Saved in:
Bibliographic Details
Main Authors: RATHNAYAKE, Darshana, WEERAKOON, Dulanga, RADHAKRISHNAN, Meeralakshmi, SUBBARAJU, Vigneshwaran, HWANG, Inseok, MISRA, Archan
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2023
Subjects:
Multi-modal interaction
3D Localization
Visual Grounding
Computer Engineering
Graphics and Human Computer Interfaces
OS and Networks
Online Access:https://ink.library.smu.edu.sg/sis_research/9221
https://ink.library.smu.edu.sg/context/sis_research/article/10176/viewcontent/Sensys2023_VGGlass_DemoPaper_CamReady.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
Description
Summary:This work demonstrates the VGGlass system, which simultaneously interprets human instructions for a target acquisition task and determines the precise 3D positions of both user and the target object. This is achieved by utilizing LiDARs mounted in the infrastructure and a smart glass device worn by the user. Key to our system is the union of LiDAR-based localization termed LiLOC and a multi-modal visual grounding approach termed RealG(2)In-Lite. To demonstrate the system, we use Intel RealSense L515 cameras and a Microsoft HoloLens 2, as the user devices. VGGlass is able to: a) track the user in real-time in a global coordinate system, and b) locate target objects referred by natural language and pointing gestures.

Similar Items