End-to-end baggage damage detection system with photogrammetry

• Main Author: Gracia, Stella
• Other Authors: Yap Kim Hui
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Electrical and electronic engineering::Computer hardware, software and systems
• Online Access: https://hdl.handle.net/10356/149571
• Institution: Nanyang Technological University

Handling damaged baggage cases is a highly frustrating experience for not only passengers but also airline staff. Beyond frustration, damaged baggage cases give rise to a significant cost to airlines every single year. In 2019 alone, there were more than 5 million damaged baggage cases worldwide and these cases cost more than $500 million of compensation cost. Currently, damaged baggage cases are handled manually by airline staff. This process is highly subjective as staff often are not sure if the damage is claimable. Some airlines also adopt technology-based solutions which include performing image classification on damaged baggage. While this solution allows the identification of damage to be automated, it is challenging to capture the full body of the baggage and thus some damage might not be captured. To address these issues, this project proposes an end-to-end baggage damage detection system with photogrammetry, named CheckBox. In the CheckBox system, two sets of videos of baggage are taken – one at departure and one at arrival. Images of the baggage will then be sampled from the videos. These images will then be stitched together with the photogrammetry technique to produce two 3D models of the baggage – one from departure and one from arrival. With a comparison algorithm, the Hausdorff distance of the two 3D models will then be computed and a damage heatmap will be generated. This metric will also be used to classify if the baggage is heavily damaged, slightly damaged, or not damaged. This classification serves as a preliminary precursor for staff to identify and prioritize urgent cases. With this approach, the damages detected are only the ones that are caused by the airline in between departure and arrival. After the backend computation is done, the information of the baggage is then passed to a web application developed with React framework. This is where airline staff can access the baggage database. The CheckBox system advances current solutions to baggage damage detection by providing a fully automated system that is able to detect damages on the full body of the baggage. The unique selling propositions of this system include its nature to be fully automated while using deterministic approaches as well as its ability to be seamlessly integrated with the current airport ecosystem.