AUTOMATED HINT GENERATION FOR INTELLIGENT TUTORING SYSTEM IN PROGRAMMING DOMAIN USING LARGE LANGUAGE MODEL

• Main Author: Christopher Swandi, Samuel
• Format: Final Project
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/82493
• Institution: Institut Teknologi Bandung

The imbalance between the number of Information Technology (IT) graduates and the demand for Information and Communication Technology (ICT) talent in Indonesia highlights the need for innovation in education. Codebuddy.ai is an intelligent tutoring system (ITS) designed to address this issue using artificial intelligence (AI). Its primary goal is to provide personalized C++ programming lessons to students. One of the key features of Codebuddy.ai is its automated hint generation, which has been proven to enhance understanding and accelerate learning. Implementing automated hint generation requires evaluating various factors, such as the foundation model, prompts, and fine-tuning, to determine the best techniques for using large language models (LLM). Additionally, due to resource limitations, deploying LLMs without directly loading the model is necessary. To deploy LLMs with limited resources, quantization is employed. This study evaluates the best techniques for creating automated hint generation using LLMs. The evaluation is conducted qualitatively using predefined metrics based on expert research. The LLM is then optimized for deployment on limited resources. The evaluation results indicate that the Phi-3 model, after prompt engineering and fine-tuning, provides accurate inference results.