TRANSFER LEARNING FOR INDONESIAN ASPECT-BASED SENTIMENT ANALYSIS USING LANGUAGE MODEL FINE-TUNING APPROACH

• Main Author: Nurul Azhar, Annisa
• Format: Theses
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/48070
• Institution: Institut Teknologi Bandung

Aspect-based sentiment analysis (ABSA) is one of the methods widely used by companies to find out public opinion in detail down to aspects contained in products or services. ABSA is typically divided into two subtasks, namely aspect extraction / categorization which aims to extract the expression of aspects / categorization of aspects into predetermined categories and sentiment classification which aims to find out the sentiment towards each aspect. Previously, research on ABSA has been done for the subtask aspect categorization and sentiment classification using convolutional neural networks (CNN) as feature extractor and extreme gradient boosting (XGBoost) as top-level-classifier (Azhar, 2019). However, the model does not generalize well in the test data and there are still a lot of out-of-vocabulary (OOV) words. Therefore, we need a new technique which could be used to solve these problems. Nowadays, the techniques of pre-training language representation models are developing so rapidly that most of the state-of-the-art results for a variety of natural language processing tasks are achieved using language representation models such as OpenAI GPT, ELMo, and BERT. In this thesis, the BERT pre-trained language model is used to complete the ASBA task for Indonesian language review texts on the hotel domain. The BERT model used is a multilingual model because currently there are no Indonesian-specific BERT pre-trained models available to the public. There are two methods of solving problems and two strategies for using language models that are compared through experiments, namely single sentence classification and sentence-pair classification as proposed in Sun, et al. (2019) as well as feature extraction and fine-tuning. Based on the results of combined experiments, the combination that produces the best performance is sentence-pair classification with fine-tuning that is equal to 0.9751. For the test results, the resulting model reaches an F1 score of 0.9765 in test data I (until December 2018) and 0.9304 in test data II (March 2019 - July 2019). There was an increase in performance of 8% in the test data I and 44% in test data II compared to the performance of the model from previous studies.