IMPLEMENTATION OF OBJECT DETECTION PERCEPTION MODULE BASED ON CAMERA SENSOR FOR AUTONOMOUS TRAM WITH YOLOV6 ON NVIDIA DRIVE AGX PEGASUS

• Main Author: Aziz, Faris
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/78055
• Institution: Institut Teknologi Bandung

Technological advancements have ushered in transformative changes across various industries, including transportation. Within this evolving terrain, trams, a fundamental mode of mass transit, have not been immune to the winds of technological evolution. Currently, the Innovative Autonomous Tram Research Project Team at ITB (Bandung Institute of Technology) is diligently crafting an autonomous, battery-powered tram, fortified with the formidable capabilities of artificial intelligence. In their quest, the team harnesses the prowess of specialized hardware tailor-made for autonomous vehicle development, NVIDIA AGX DRIVE Pegasus. The Innovative Autonomous Tram Research Project Team at ITB comprises multiple teams, with one of them being the perception team. The perception team is responsible for developing algorithms that perform object detection. The results of this detection are then sent to the decision-making team in the form of information, including bounding box data, coordinates (x and y), speed, and distance from the tram. The architecture of the object detection model must have faster inference speeds compared to the previous YOLOv3 architecture to prevent delays in decision-making. Additionally, the implemented object detection algorithm must be capable of detecting objects commonly found in the Indonesian environment. The need for quick inference speed takes precedence over the confidence level of object detection because rapid decision-making is crucial for accident prevention. In this research, it was discovered that YOLOv6 is an architectural model with a significantly shorter inference time, precisely 35 ms, and can be converted into a format supported by NVIDIA DRIVE AGX Pegasus. Fine-tuning enables the detection of local Indonesian objects. Fine-tuning was performed on 1600 augmented data points, resulting in 3200 data points. These were then divided with a composition of 60% for training, 20% for validation, and 20% for testing. After fine-tuning, the results showed that a learning rate set at 0.0128 achieved an mAP@0.50 of 0.8662 and an mAP@0.50:0.95 of 0.6918. Fine-tuning was carried out over 50 epochs, resulting in an mAP@0.50 of 0.7715 and an mAP@0.50:0.95 of 0.5832. The perception module can be integrated among sensors and between sensors by applying naming function conventions and adjusting arguments when running the program. Detection results between sensors were achieved using a modified NMS algorithm to adapt to speed and distance measurements.