Development of human-machine interface for mixed-swarm robot team

• Main Author: Nur Amirah Binte Rhymie
• Other Authors: Li King Ho Holden
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Mechanical engineering
• Online Access: https://hdl.handle.net/10356/167647
• Institution: Nanyang Technological University

A HMI wearable with control scheme was developed to interface with a MSRD element for high intensity military surveillance operations in urban environments. Research reveals that MSRD technology is being conceptualised for real-world applications, and computer vision frameworks are utilised to detect and track targets. A decision matrix was used to decide on the eventual HMI and MSRD element, by considering factors that influence development and on-the-ground use. A computer vision framework was also implemented to simulate surveillance and threat detection. Eventually, an arm guard with HMI controls and a JETANK were developed to provide the human operator with real-time information and increased awareness via motion detection algorithm and autonomy over decision-making. The system was evaluated through experiments which involved a circuit to navigate the JETANK with a gesture sensing control scheme using APDS9960 module and employing test objects to analyse detectability rates for the motion detection algorithm. State-of-the-art gesture sensing using computer vision framework has a success rate of 91% as researched by Anand et al, and the control scheme achieved standard success rates. Motion detection algorithm results also produced comparative success rates to Anand et al’s study, suggesting that the algorithm is reliable in detecting camouflaged moving objects. Future developments to the system may include experimenting with the optimal latency for the APDS9960 module and integrating additional features to the motion detection algorithm to further improve the system’s performance. Overall, the presence of a human operator in MSRD teams might be able to provide better situational awareness and operational performance with further developments.