Use of world model in (simulated) robotic motion

• Main Author: Shen, Chen
• Other Authors: Zinovi Rabinovich
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/148065
• Institution: Nanyang Technological University

This project aims to implement and experiment with the World Models architecture in a simulated robotics environment. The World Models architecture consists of three main stages: learning a way to represent the world environment in a compressed format, learning the dynamics of the environment with respect to the compressed format, and finally learning a control strategy to maximise reward in the environment. The goal of these strategies combined is to train a well performing policy with respect to a chosen environment. This project seeks to test this architecture in a simulated robotics environment, specifically, a 3D racing scenario similar to the 2D CarRacing-v0 environment provided by Gym. This provides insight into the potential real-world applications of the World Model architecture in robotics and beyond. The final developed environment is based on the PyBullet physics engine and the architecture was tested in two similar environments with different observation perspectives: a control top-down view similar to the original CarRacing-v0, and a first-person camera view with the camera mounted to the car. The experiments showed no significant degradation of performance when switching from the top-down observation perspective to the first-person perspective, which implies that the World Model architecture is able to generalise to more realistic environment observations. This shows the promise of World Models in robotics reinforcement learning applications and the need for possible further exploration into higher fidelity simulations or even further testing in the real world.