Deep learning-based car plate optical character recognition

• Main Author: Choo, Zhen Bo
• Format: Final Year Project / Dissertation / Thesis
• Published: 2022
• Subjects: TK Electrical engineering. Electronics Nuclear engineering
• Online Access: http://eprints.utar.edu.my/4953/1/3E_1806581_Final_report_%2D_ZHEN_BO_CHOO.pdf; http://eprints.utar.edu.my/4953/
• Institution: Universiti Tunku Abdul Rahman

In the field of intelligent transport systems, recent years have witnessed the application of deep learning techniques to both car plate detection and recognition. The latter stage, known as optical character recognition (OCR), is more challenging as it requires an accurate prediction of the entire license numbers. One of the widely used OCR engines is the Tesseract, which uses long short-term memory (LSTM). However, the drawback of this approach is the time-consuming image preprocessing techniques. This project aims to design an accurate yet lightweight OCR solution by exploring the bidirectional LSTM, connectionist temporal classification (CTC) and ResNet. The training datasets comprise two public synthetic datasets and one self-collected dataset, which is specific to the Malaysian car plate format. The trained models are subsequently optimized via OpenVINO for faster inference time. Results show that the proposed solution is 10x faster than the Tesseract OCR while still having more than a 2x increase in accuracy. In a case study of vehicle surveillance, a local webserver is established to host the newly developed OCR solutions in combination with a pre-trained YOLOv4 car plate detection. Results show that the end-to-end solution can process video streams at a rate of 20 frames per second (FPS).