DEVELOPMENT OF CHATBOT MODULE IN AN INTELLIGENT TUTORING SYSTEM FOR ENGLISH LANGUAGE LEARNING USING LARGE LANGUAGE MODEL

• Main Author: Dhia Rafi, Ziyad
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/82400
• Institution: Institut Teknologi Bandung

Based on research findings, Indonesia has low English proficiency, worsen by a lack of teaching resources and educational inequality. Therefore, in this final project, a solution is developed in the form of an Intelligent Tutoring System (ITS) for English learning using a Large Language Model with few-shot prompting. The chatbot module of the ITS is developed to create personalized learning and enhance the effectiveness of the learning process. This chatbot module is implemented as a backend application that accommodates various interactions with the chatbot. The development of the chatbot began with an experiment comparing basic LLMs to find the best base model. In this experiment, various base model candidates were applied with different prompts, and each combination was tested for output quality. The base model was also developed into 8 different use cases, namely student QA, feedback generation, translation, grammar correction, tutor QA, question generation, answer generation, and explanation generation. The use case development and model performance improvement used the few-shot prompting technique by providing a few examples of tasks. From the model comparison experiment, the best model obtained was llama 3 7B instruct, with a simple task instruction prompt type. Following the performance enhancement experiment, there was a 7.97% improvement in the base model's performance, with a final informativeness score of 0.958 and an accuracy score of 0.9649. The model was developed into an English learning chatbot integrated with the Intelligent Tutoring System application in the form of an API. The results of functional testing indicate that the chatbot module has functioned as intended.