Intelligent robot grasp planning with multimodal large language model - Liu, Songting (Nanyang Technological University, 2024)
Autonomous robot grasping in multi-object scenarios poses significant challenges, requiring precise grasp candidate detection, determination of object-grasp affiliations, and reasoning about inter-object relationships to minimize collisions and collapses. This research presents a novel approach to address these challenges by developing a dedicated grasp detection model called GraspAnything, and integrating it with a multimodal large language model (MLLM) for open-vocabulary object detection and grasp sequence planning.