A STATISTICAL AND RULE-BASED GRAMMAR CHECKER FOR INDONESIAN TEXT

• Main Author: FAHDA (NIM : 13513079), ASANILTA
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/21279
• Institution: Institut Teknologi Bandung

Spelling and grammar errors in Indonesian text are not an uncommon occurrence, even in formal contexts such as academic or bureaucratic documents. Meanwhile, the use of proper language is essential for expressing ideas and thoughts clearly in written text. Spelling and grammar checkers are widely-used tools which aim to help in detecting and correcting various writing errors. However, there are currently no proofreading systems capable of checking both spelling and grammar errors in Indonesian text. Therefore, this study proposes an Indonesian spelling and grammar checker prototype which uses a combination of rules and statistical methods. <br /> <br /> <br /> There are currently 38 rules from regular expressions which detect, correct, and explain common errors in punctuation, word choice, and spelling. The spelling checker then examines every word using a dictionary trie to find misspellings and Damerau-Levenshtein distance neighbors as correction candidates, as well as morphological analysis for processing certain word forms. A bigram or co-occurrence-based Hidden Markov Model is used for ranking and selecting the candidates. The grammar checker uses a trigram language model from tokens, POS tags, or phrase chunks for identifying sentences with incorrect structures according to a threshold value chosen empirically. <br /> <br /> <br /> By experiment, the co-occurrence HMM with an emission probability weight coefficient of 0.95 and transition probability weight coefficient of 0.05 is selected as the most suitable model for the spelling checker. As for the grammar checker, the phrase chunk model which normalizes by chunk length and uses a threshold score of -0.4 gave the best results. The parameter values achieving the best results are applied in the final system. The document evaluation of this system showed an overall accuracy of 83.18% and the prototype is implemented as a web application.