TRANSLATION OF INDONESIAN SIGN LANGUAGE JAKARTA'S DIALECT WORDS USING MACHINE LEARNING

• Main Author: Avicena Ismunandar, Abiyyu
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/64106
• Institution: Institut Teknologi Bandung

ABSTRACT TRANSLATION OF INDONESIAN SIGN LANGUAGE JAKARTA'S DIALECT WORDS USING MACHINE LEARNING Oleh ABIYYU AVICENA ISMUNANDAR NIM: 13517083 Bahasa Isyarat Indonesia (Bisindo) is the language that are used by the deaf communities in Indonesia. Each region in Indonesia has their own unique sign language and for this research the Jakarta’s dialect will be translated. A translator is needed to help and ease communication between people who uses Sign language and does not. To create the translator a machine learning model needs to recognize words in Bisindo Jakarta’s dialect. However, currently the database for Bisindo Jakarta’s dialect is not available, therefore, Bisindo Jakarta’s dialect data needs to be collected. This research has managed to collect 549 videos with the help of volunteers from the writer’s colleagues, family, and Pusbisindo volunteers. Words that were collected from the volunteers are 20 words that are used on day-to-day basis by signers. The machine learning model architecture that is used in this research is a combination of the CNN and RNN architecture. To help the feature extraction of the sign videos a pose estimation that extract body landmarks called Blazepose is utilized. The translator model that recognizes 20 words reached an F1-score of 0.3067. This research is a start to bridge the communication gap between signers and non-signers in Indonesia. Keywords—Bisindo; Machine Leaarning; Translator