OCCLUDED OBJECT DETECTION WITH INFRARED CAMERA ON MOBILE ROBOT FOR SEARCH AND RESCUE

• Main Author: Kane, Hansel
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/57723
• Institution: Institut Teknologi Bandung

Disaster effects tend to be more severe in developing countries. Improving the disaster response system is the key to prevent casualties. Therefore, a system that could hasten search and rescue processes is utterly necessary. Mobile robot is a really suitable candidate for such task. Mobile robot in this research has a thermal camera equipped as thermal cameras tends to be unaffected by lighting variation, thus providing more robust detection. The models used MobileNetv2 architecture to detect human in lying position. Models were trained using infrared images containing human in lying position representing a casualty. Three variation of training data that result in three different models (Mnol, Msebagian, Msatu) with different number of dataset, were used to observe the effect of partially occluded augmentation training data. Furthermore, four category of tests were carried out to observe the performance of the models: unoccluded, partially occluded, fully occluded human and human in construction area. The result of unoccluded test were 0.65, 0.8, 0.74 in terms of mAP for Mnol, Msebagian, and Msatu respectively. While the result of partially occluded test were 0.89, 0.86, 0.92 in terms of mAP for Mnol, Msebagian, and Msatu respectively. The result of fully occluded test were 0.95, 0.95, 0.97 in terms of mAP for Mnol, Msebagian, and Msatu respectively. Moreover, loss of the three models converged to 4/7. The result of human detection in construction area were 0.75; 0.9; 0.87 for Mnol, Msebagian, and Msatu respectively. Modifying training dataset by adding random-generated occluded images improves model’s performance in general as shown by Msatu and Msebagian that yields higher mAP than Mnol in most of the categories. Model MobileNetv2 had reached its full capacity, indicated by a decrease in mAP score when quantity of augmentation datas was increased (the mAP of Msatu is lower than Msebagian). Furthermore, temperature measurement is also carried out yielding 32 to 34 degree Celcius for human skin detected by Lepton 3.5. A live-stream is performed to test the speed of MobileNetv2. Deployed on Raspberry Pi 3b+, MobileNetv2 could run in around 2 fps. All three models successfully detect a lying person in unoccluded, partially occluded, and fully occluded condition while also performing a real-time temperature measurement of the human skin. Various materials were analyzed in terms of its thermal transmittance which varies when extra layers were added. Materials are sorted in terms of its thermal transmittance in descending order: paper, plastic, cotton, denim.