EXPLORING CHATGPT-ASSISTED FEEDBACK FOR STUDENTS' WRITTEN RESPONSES TO MATHEMATICS PROBLEMS

• Main Author: Kurnia Rahmawati, Rikha
• Format: Theses
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/87943
• Institution: Institut Teknologi Bandung

In this project, the author explores the use of ChatGPT to provide feedback on students' written responses to mathematics problems. Two groups of students were involved: Grade 7 students and Grade 10 students. Feedback was generated after training ChatGPT using prompt engineering techniques, including chain-of-thought prompting, one-shot learning, and human-in-the-loop approaches. This exploration was conducted in five stages following the ADDIE research model (Analyze. Design, Develop, Implement, and Evaluate). In the first stage, an analysis was conducted to determine the ChatGPT model and the practice problems to be used. In the second stage, the author designed prompts to iteratively train ChatGPT to produce feedback. In the third stage, the author observed the performance of the designed prompts and made improvements to them. In the fourth stage, students were given a practice problem, and their answer sheets were collected. Feedback was then provided for ten responses from the Grade 7 group and eleven responses from the Grade 10 group by both ChatGPT and the mathematics teachers of the respective groups. In the fifth stage, the mathematics teachers from both groups, along with six other teachers, were asked to evaluate ChatGPT's feedback by assigning scores based on a Likert scale (1-4). They were also asked to complete a questionnaire regarding the feasibility and accuracy of ChatGPT's feedback. Two versions of feedback were then distributed to each student. Students were asked to assess the clarity and usefulness of each type of feedback and were also tested on their ability to identify which feedback was generated by ChatGPT. The evaluation of ChatGPT’s feedback by teachers showed positive tendencies, although they noted that some aspects still needed improvement. Teachers recognized the need to improve the quality of their own feedback after reviewing the feedback generated by ChatGPT. Most students rated ChatGPT's feedback as clear and useful. Students’ ability to identify ChatGPT-generated feedback was relatively low, indicating that ChatGPT's feedback appeared natural and was difficult to distinguish from teacher-generated feedback. Overall, ChatGPT's feedback was deemed suitable for use, with the note that its effective implementation requires supervision and additional training by teachers.