Reinforcement learning based motion planning of quadrotors using motion primitives

• Main Author: Efe Camci
• Other Authors: Chen I-Ming
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Electrical and electronic engineering::Control and instrumentation::Robotics; Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/137183
• Institution: Nanyang Technological University

Motion planning of robots in real world is challenging due to the uncertainty in environments and robot models, the computation and sensing limitations on hardware, and the complexity of the tasks to be performed during operations. Motivated by these problems, the main contribution of this thesis is a Reinforcement Learning (RL) based approach for motion planning of quadrotors that can deal with uncertainties, work with rudimentary hardware, and minimize expert user intervention during complex operations. Aligned with the hierarchical motion planning pipeline of quadrotors, i.e., high-level, position, and orientation planning, we propose a novel planning framework at each domain based upon RL. In Chapter 3, we propose a planning framework in orientation domain which allows a quadrotor to plan swift maneuvers and decrease navigation time in dense environments. Our RL agent learns the complex relations between abrupt control inputs and the motion of a quadrotor incorporating probable uncertainties such as the unmodeled dynamics during agile maneuvers. It then utilizes this knowledge during deployment to plan relatively swift and reasonably precise maneuvers. In Chapter 4, we propose a planning framework in position domain which caters for navigation in unstructured environments using raw data obtained from onboard sensors. Our RL agent maps instantaneous raw sensor data to local motion plans in the position domain which cater for obstacle avoidance and navigation towards goal simultaneously. It learns essential navigation policies which are transferable from virtual environments to previously unseen real environments. In Chapter 5, we propose a planning framework in high-level user input domain which creates aesthetically aware motions of a filming quadrotor that follows an actor and shoots its video. Trained using both a handcrafted and a human reward, our RL agent extracts the intricate relations between video scene context and aerial camera viewpoints. Accordingly, it creates high-level viewpoint sequences which govern low-level position planning of a filming quadrotor for shooting better video clips. We evaluate and validate our RL-based planning frameworks by conducting virtual and real flight tests extensively. We discuss our methods in comparison with the current state-of-the-art methods. In the end, we share the lessons learned during this study and indicate a few promising directions for future work.