Handwritten mathematical expression recognition

• Main Author: Ang, Brian Meng Hong
• Other Authors: Loke Yuan Ren
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/166072
• Institution: Nanyang Technological University

Handwritten Mathematical Expressions Recognition (HMER) is a crucial problem in the field of artificial intelligence and machine learning, given the complexity and variability of handwriting in two dimensions. Existing approaches to HMER face challenges such as handwriting style variability, non-standard symbols and notation, and errors and ambiguities in writing. In this study, we propose a novel approach to HMER using rotary position embeddings and a hybrid loss calculation of connectionist temporal classification and cross entropy to improve the accuracy of transformer-based models for recognizing cursive handwriting and complex equations. We train and test our approach on public databases from CHROHME 2014, 2016, and 2019 of offline HMEs. Our experiments demonstrate that our approach results in higher expression recognition rates and lower word error counts compared to existing approaches. Notably, our results are comparable to recent studies in the field, highlighting the potential of our approach to advance the state of the art in HMER.