Learning a closed-loop policy for scooping task

• Main Author: Lau, Wei Quan
• Other Authors: Ang Wei Tech
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering; Assistive feeding; Scooping; Behavioural cloning
• Online Access: https://hdl.handle.net/10356/177105
• Institution: Nanyang Technological University

Assistive feeding technologies can play a crucial role in improving the quality of life for individuals with disabilities or age-related limitations who struggle with self-feeding. Scooping task is important to enable assistive feeding. Therefore, a Dynamic Motion Primitives (DMP) based policy has been developed at Rehabilitation Research Institute Singapore (RRIS) for food scooping. Existing assistive feeding systems including the one developed at RRIS mostly use an openloop policy for scooping. This makes it hard to control the amount of food to be scooped which is important as many times therapists have requirements for people with disabilities to have a specific bite-size. Having a closed-loop policy which can adapt the scooping trajectory based on the sensor feedback such as camera, force torque sensors can address this limitation. Developing a closed-loop policy for scooping task which involves dealing with deformable objects like food is not trivial as there is a lack of physics models, which makes classical planning approaches difficult. Hence, data-driven approaches are used which collect training data through demonstrations and learn a policy using that data. However, for learning a policy using data-driven approaches, data specific to a given robot and task needs to be collected. The data can be collected in simulation when a large amount of data is needed or on the real arm if a small amount of data is needed. Therefore, in this work, we aim to learn a closed-loop scooping policy and execute it on the xArm6 at RRIS. It is to noted that learning a closed-loop policy that can scoop a specified amount of food is out of scope of this project as it requires advanced Reinforcement Learning and will be explored in the future by RRIS researchers. The scope of this project is to develop a framework for learning a closed-loop policy and demonstrate that the framework works by learning a simple Behavioural Cloning based policy using that framework. This ensures that this framework can be used in future by RRIS researchers for developing more advanced closed-loop policies. We create the framework for both the simulated and real arm. For simulation, we survey various simulators and recommend one that is compatible with xArm6 and deformable object manipulation. From our experiments, we demonstrate that the policy learned in simulation can generate a scooping trajectory on simulation and the policy learned from the real arm can scoop the food up on the real arm. However, there is a sim-to-real gap when using the learnt policy from simulated data on the real arm.