Development of virtual environment for bimanual robotic manipulation for industrial tasks

• Main Author: Shome Mrigank
• Other Authors: Domenico Campolo
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering; Bimanual robotic manipulation; Virtual environment; Pose estimation accuracy; Bimanual robotics industrial tasks; Robotics; Bimanual robots
• Online Access: https://hdl.handle.net/10356/177507
• Institution: Nanyang Technological University

Bimanual robotic manipulation involves the coordinated usage of two robotic arms simultaneously, and it offers enhanced productivity and flexibility in various applications, including industrial tasks. Implementation of a system like this presents considerable challenges including coordination between the two robotic arms, accurate modelling of physical interactions, and the development of a robust control strategy. Virtual environments can tackle these challenges by providing the ability to evaluate potential limitations before the physical system is built and deployed. This project aimed to develop a virtual environment for testing of user-defined scenarios for bimanual robotic operations, and utilise a suitable physics engine for understanding the contact dynamics between the end effectors of the robotic arm and the manipulated object. The project investigated the impacts of factors such as mechanical inaccuracies, non-synchronised contact, initial object orientations, object pose estimation errors, and changes in the object geometry on the bimanual robotic operation dynamics. All the while maintaining accessibility with good computational efficiency. The results of the investigation quantified the sensitivity of the system to the aforementioned factors and identified boundaries within which predictable contact between the object and robotic arm end effectors is possible. A significant contribution from this work is the development of the fitted ellipse method using singular value decomposition which optimises the initial positions of the robotic handles. This method considers the specific shape and dimensions of the object which is being manipulated, and resulted in lower deviation between the final and initial positions of the manipulated object compared to the unfitted circle method, upon contact. The project’s findings provide tangible insight into the dynamics of bimanual robotic manipulation and improving the efficiency of such systems in an industrial domain. There are limitations such as using a decoupled control strategy and non-complex objects, which creates the direction for future work on this project, and also an extension to a 3D virtual environment.