Development of vision-based terrain recognition method for lower limb exoskeleton for outdoor environments

• Main Author: Gupta, Paluk
• Other Authors: Ang Wei Tech
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Engineering; Physics; Terrain recognition; Lower limb exoskeleton; Outdoor environments; Deep learning; Convolutional neural networks (CNN); 3D point cloud data; Depth camera; ZED camera; Vision-based systems; Mobility assistance; Image processing; Depth inpainting; Assistive technology; Machine learning; Terrain classification; Adaptive mobility; Exoskeleton technology; Environmental adaptation; Real-time processing
• Online Access: https://hdl.handle.net/10356/177580
• Institution: Nanyang Technological University

This report details the journey made in creating an improved vision-based terrain recognition system for a lower limb exoskeleton. This robotic prosthesis helps people move around by navigating indoor and outdoor terrains by giving them torque assistance and leg support. It aids those with disabilities or paralysis in their lower limbs, allowing them to achieve mobility on their own. To navigate through different terrains, the exoskeleton requires a thorough perception system to be able to classify the environment around to execute terrain specific movements. The project's key part involves enhancing an inventive method for classifying terrains which can identify seven different types of surfaces: level ground, stairs (up and down), slopes (uphill and downhill), obstacles and gap. The heart of the system is a deep learning model, specifically Convolutional Neural Network (CNN). It processes 3D point cloud data from a depth camera. The main purpose of this model is for terrain classification by estimating terrain features from binary images. One problem that can be seen in using exoskeletons for everyday activities comes when vision-based systems fail due to conditions such as occlusions in depth maps caused due to uneven texture of surfaces, sunlit or reflective surfaces - these factors greatly affect image quality and how accurately they are classified. This project addresses these issues through a few key improvements. Firstly, the report details transitioning from using a Realsense camera to a ZED camera. The ZED camera has a polarizer built in which makes it more suitable for enhancing point cloud capture in different settings. Additionally, it examines the idea of other techniques like getting an area of interest from depth map, employing neural mode present in ZED Camera and depth inpainting for bettering terrain classification precision under difficult terrain circumstances. These methods mentioned ensure improved depth maps are created or missing depth values are filled, especially useful for data gaps resulting from distorted depth captured from the camera. Additionally, they construct point clouds on reflective or transparent areas. Finally, the report concludes with examining different shaped obstacles and the algorithm’s accuracy in classifying them with the goal to improve the model by training it on obstacles with different characteristics. The completion of this project marks an important advancement in making exoskeletons usable for various environments, which will lead to more widespread use and application of lower limb exoskeletons in daily life.