Arbitrary-view human action recognition for human robot interaction

• Main Author: Loh, Zachariah Jin Jun
• Other Authors: Li King Ho, Holden
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering; View
• Online Access: https://hdl.handle.net/10356/177242
• Institution: Nanyang Technological University

This report presents the development and evaluation of an innovative Arbitrary-view Human Action Recognition (AVHAR) system, utilising the Robomaster Tello Talent for aerial camera feeds. The aim is to enhance capabilities in human-robot interaction by addressing the challenge of recognising human actions from any viewpoint, the project leverages artificial intelligence, with a focus on Long Short-Term Memory (LSTM) networks.The objective of this project is to develop a system capable of processing skeletal data with high accuracy, achieving real-time action recognition. The evolution of AVHAR has been marked by several challenges, notably the variability of human actions when observed from different viewpoints. Occlusions also present a persistent challenge, where parts of the skeletal structure may be obscured by objects or other individuals, complicating accurate recognition. The project will focus on integrating TensorFlow, OpenCV, MediaPipe, and other libraries to build a comprehensive software infrastructure capable of extracting key points, preprocessing data, and applying sophisticated machine learning algorithms for action detection. This foundation facilitated the exploration of LSTM models, enhanced by three dense layers for refined data analysis and prediction accuracy.