Transfer learning for a robotic arm to pick up wheel bearings using deep neural networks

• Main Author: Lim, Andres
• Other Authors: Soh Yeng Chai
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Electrical and electronic engineering::Computer hardware, software and systems; Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision
• Online Access: https://hdl.handle.net/10356/158404
• Institution: Nanyang Technological University

Machine learning algorithms provide a flexible alternative to explicitly programming a computer to perform a specific task, in the form of neural networks that can learn to give a correct answer after being trained on a set of inputs and their corresponding correct answers. One such problem may be identifying an object’s position from a given image for manipulation by a robot arm; it would be impossible to explicitly define in a program the correct output for every possible input. Convolutional neural networks are types of machine learning algorithms meant to handle visual input data by gathering spatial information about the input pixels and identifying features from them. At present, machine learning is highly specific in terms of both the inputs that can be accepted and the output generated. Most trained neural networks cannot be easily repurposed to solve other problems. Training a brand new network from scratch to solve new problems is a time-consuming and data-intensive process, especially for convolutional neural networks which have many components. Transfer learning aims to improve the training speed and generalizability of neural networks by capitalizing on knowledge already learnt by existing neural networks from solving a certain problem, and transferring it into another network meant to solve a different but relevant problem. This project explores the transfer of learning between convolutional neural networks trained to localize wheel bearing hubs. The predictions of these models will be provided to a motion planner that controls a robot arm to pick up the items in a Unity simulation. While models that employed transfer learning did show some improvement in performance over a baseline, and are, to a small extent, able to generalize to pick up wheel bearing hubs they are not trained to pick up, such improvements are insufficient to make up for other problems such as lack of training data or poor grip. It was found that while models aim to reduce their RMS translation error during training, this is not a guarantee that the trained model will have good performance. Moreover, more improvements are needed before these neural networks are able to perform well in the real world. Most important of all, test time input images must closely resemble the training data for the neural networks to display good performance.