Nao robot kinect control

• Main Author: Cumming, Matthew
• Other Authors: Song Qing
• Format: Final Year Project
• Language: English
• Published: 2014
• Subjects: DRNTU::Engineering
• Online Access: http://hdl.handle.net/10356/60415
• Institution: Nanyang Technological University

This is the Final Year Project report on producing an application to control Aldeberan Robotics’ Nao Robot using Microsoft’s Kinect sensor. The process of developing this application involved learning how to program the Kinect and the Nao robot as well as researching and designing artificial neural networks and machine learning algorithms for gesture recognition. The majority of the time spent working on this project involved research, and learning how to use the tools necessary to build the final application. This project was an open ended one; once an interface has been created the robot could be controlled via the Kinect in any desired manner. A WPF application was created in Visual Studios, using C#, to take information from the Kinect about the user’s body position and output it. The outputs were analysed and used with python scripts to record gestures from the Kinect. These recorded gestures were then used to train a multilayer perceptron artificial neural network (also programmed in python) using a machine learning algorithm to recognise various gestures. Once the program had learned the gestures, it could perform gesture recognition in real time and control the robot accordingly. Further modes of operation were developed whereby the robot would imitate the user; moving to the same position and orientation as the user in its own environment. This project was successful, providing a great deal of research and learning opportunities while also establishing a system that can fully control the Nao robot through any interaction with the Kinect. It would be easy to develop this into a powerful tool for teaching Nao new behaviours or skills via human demonstration.