Smart cashierless checkout system for retail using machine vision

• Main Author: Lee, Ren Yi
• Format: Final Year Project / Dissertation / Thesis
• Published: 2022
• Subjects: TJ Mechanical engineering and machinery
• Online Access: http://eprints.utar.edu.my/5393/1/MH_1702805_Final_%2D_REN_YI_LEE.pdf; http://eprints.utar.edu.my/5393/
• Institution: Universiti Tunku Abdul Rahman

As Corona Virus 2019 (COVID-19) pandemic strikes the world, retail industry has been severely impacted especially in its daily operation due to the restriction of workforce and most of the face-to-face services, including checkout, are associated with high risk of developing spread chain of COVID-19 virus. Despite there are multiple computer vision-based solutions available in the field such as on-shelf checkout and sensor fusion, but they can be expensive and may require overhaul of stores, which is unfeasible for small retail stores. Therefore, a software prototype of intelligent cashierless checkout system is proposed to help small-scale retail stores in minimizing the risk of developing COVID-19 virus spread chain as well as the workforce requirement during checkout using state-of-the-art object detection models. This project was performed in 2 parts where the first stage involved an image synthesis algorithm to automatically produce visually realistic product images using Generative Adversarial Network (GAN). Several GAN architectures such as CycleGAN and AttentionGAN were studied and compared in terms of their effectiveness in generating realistic shadow in actual checkout scenario. CycleGAN results in more realistic shadow with Fréchet inception distance (FID) of 40.99. In the following stage, a publicly available dataset, MVTec D2S dataset were used to benchmark multiple object detection models used for product recognition. By using You Only Look Once (YOLO) v5L as the baseline model, several improved models were developed by replacing the backbone structure with other light-weight architectures to improve computation efficiency when deployed on edge devices. After training the model with dataset generated in previous stage, the proposed model with MobileNet V3 surpassed baseline model in terms of inference time, with only 0.142s while maintaining high Mean Average Precision (mAP) of 98.2% and Checkout Accuracy (cAcc) of 89.17% on Jetson Nano.