Sign language recognition using deep learning through LSTM and CNN

• Main Authors: Kiran, Pandian, Mohd Azraai, Mohd Razman, Ismail, Mohd Khairuddin, Muhammad Amirul, Abdullah, Ahmad Fakhri, Ab. Nasir, Wan Hasbullah, Mohd Isa
• Format: Article
• Language: English
• Published: Penerbit UMP 2023
• Subjects: QA76 Computer software; TJ Mechanical engineering and machinery
• Online Access: http://umpir.ump.edu.my/id/eprint/38079/1/Sign%20Language%20Recognition%20using%20Deep%20Learning%20through%20LSTM%20and%20CNN.pdf; http://umpir.ump.edu.my/id/eprint/38079/; https://doi.org/10.15282/mekatronika.v5i1.9410; https://doi.org/10.15282/mekatronika.v5i1.9410
• Institution: Universiti Malaysia Pahang

This study presents the application of using deep learning to detect, recognize and translate sign language. Understanding sign language is crucial for communication between the deaf and mute people and the general society. This helps sign language users to easily communicate with others, thus eliminating the differences between both parties. The objectives of this thesis are to extract features from the dataset for sign language recognition model and the formulation of deep learning models and the classification performance to carry out the sign language recognition. First, we develop methodology for an efficient recognition of sign language. Next is to develop multiple system using three different model which is LSTM, CNN and YOLOv5 and compare the real time test result to choose the best model with the highest accuracy. We used same datasets for all algorithms to determine the best algorithm. The YOLOv5 has achieved the highest accuracy of 97% followed by LSTM and CNN with 94% and 66.67%.