Template-based category-agnostic instance detection for robotic manipulation

• Main Authors: Hu, Zhongxu, Tan, Runjia, Zhou, Yanxin, Woon, Junyang, Lv, Chen
• Other Authors: School of Mechanical and Aerospace Engineering
• Format: Article
• Language: English
• Published: 2023
• Subjects: Engineering::Mechanical engineering; Robots; Solid Modeling
• Online Access: https://hdl.handle.net/10356/164360
• Institution: Nanyang Technological University

An intelligent robotic system is one of the key pillars of a smart factory that requires flexibility to handle a variety of tasks. Perception is a key enabling technology for robots. Most existing object detection studies have mainly focused on category-specific objects and have achieved impressive performance. However, robotic systems, particularly in industrial scenarios, typically interact with many category-agnostic objects, which the robot must detect instantly without pre-training. Therefore, in this study, we proposed a template-based detection and segmentation approach, which incorporated a multi-level correlation model and a similarity-refine module, for handling the category-agnostic instance. The proposed approach was then validated and demonstrated in an interactive and adaptive robotic application scenario designed for the typical pick-and-place task. Among them, the picking scan path and location were instructed through human guidance with hand tracking. The neural rendering technology was also introduced to render novel views of the template. The proposed approach was evaluated using a benchmark and verified through a real demonstration.