Recognizing Filipino sign language video sequences using deep learning techniques

• Main Author: Tupal, Isaiah Jassen L.
• Format: text
• Language: English
• Published: Animo Repository 2023
• Subjects: Philippine Sign Language—Data processing; Sign language—Data processing; Artificial intelligence
• Online Access: https://animorepository.dlsu.edu.ph/etdm_ece/25
• Institution: De La Salle University

The use of machine intelligence for sign language is a popular method, especially in the field of sign language recognition. A model that performs well on SLR can bridge the gap between the deaf and the hearing. Furthermore, it can be applied to a great number of applications such as sign language education. The overarching issue in this field is the chronic lack of high-quality and large-volume datasets. This holds back SLR research, especially on less studied sign languages such as FSL. In addition, the inclusion of real-world applications is rare in FSL research. With this in mind, the study focused on creating a high-quality FSL dataset, building an accurate model, as well as developing an application to deal with the current gaps in FSL research. A dataset of more than 2000 video clips with 105 different signs was made. The signs were carefully chosen by an expert to make sure that these are useful introductory signs. Each frame of the video contains joint locations which were extracted by MediaPipe. Graph convolutional networks and Gated Recurrent Networks were then used to classify the signs. Using the model, an FSL E-learning desktop app was then developed. The top model, MediaPipe-GRU, was able to achieve a 100% top-5 accuracy on the dataset.