Investigation of adopting large language model to generate robotics control based on designed prompt

• Main Author: Pan, Junyu
• Other Authors: Xie Lihua
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering
• Online Access: https://hdl.handle.net/10356/177119
• Institution: Nanyang Technological University

This project aims to explore the effectiveness of using large language models to generate control instructions for robotics applications. Recent advancements in deep learning techniques have led to the development of Large Pre-Trained Language Models like GPT-3 and GPT-4, which have shown great results in generating high-quality texts in a wide range of applications, robotics is an area where language models can be applied to generate control instructions for robots based on specific prompts. Hence, identifying a suitable robotics platform and designing a prompt that can be used to generate robotics instructions, helps to explore the comparability with each other. The experiment was conducted using CoppeliaSim, a robust simulation platform, and Visual Studio Code for scripting the prompt and integration of OPENAI’s GPT 4 model and CoppeliaSim, allowing real-time communication between them. The core of the research involved developing a framework that enables the robot to send descriptive prompts to ChatGPT based on its sensory inputs, and in return, receive navigational commands that are executed within the simulation. This paper presents the methodology, setup, and execution of the experiment, highlighting the innovative use of ChatGPT in robotic navigation. The results demonstrated the potential of LLMs to revolutionize how robots understand and interact with their environment, paving the way for more intuitive human-robot interactions and enhanced decision-making capabilities in unstructured settings.