Exploring Confidence Accuracy and Item Difficulty in Changing Multiple-Choice Answers of Scientific Reasoning Test

• Main Authors: Fadillah, Sarah Meilani, Ha, Minsu, Nuraeni, Eni, Indriyanti, Nurma Yunita
• Format: Article
• Language: English
• Published: Universiti Utara Malaysia Press 2023
• Subjects: L Education (General)
• Online Access: https://repo.uum.edu.my/id/eprint/29676/1/MJLI%2020%2002%202023%20319-341.pdf; https://doi.org/10.32890/mjli2023.20.2.5; https://repo.uum.edu.my/id/eprint/29676/; https://e-journal.uum.edu.my/index.php/mjli/article/view/17901; https://doi.org/10.32890/mjli2023.20.2.5
• Institution: Universiti Utara Malaysia

Purpose – Researchers discovered that when students were given the opportunity to change their answers, a majority changed their responses from incorrect to correct, and this change often increased the overall test results. What prompts students to modify their answers? This study aims to examine the modification of scientific reasoning test, with additional exploration on confidence accuracy and its relation to item difficulty. Methodology –A set of pre-test and post-test experiments which included 20 items of scientific reasoning test with confidence judgement on each item were used. The items of the instruments were assessed for their validity by analysing their psychometric properties using the three-parameter (3PL) Item Response Theory, which was carried out in R studio. The set of items were randomly administered to 205 Indonesian undergraduate students with a background in science education related major. The accuracy of confidence was determined by categorising correct or incorrect answers to scientific reasoning questions based on their level of confidence. Findings –The results revealed that responses were modified more frequently from incorrect to correct than from correct to incorrect, resulting in a significant gain in overall scientific reasoning score although these modifications were not shown to be connected to the item’s difficulty level. Even though confidence level also increased significantly, it was observed that Indonesian students repeatedly responded with overconfidence even after sitting for the same test after three weeks, which could indicate a lack of metacognitive ability. The findings of this study serve to spur educators to begin actively engaging in metacognitive training in their teaching and learning activities as a result of overconfidence that frequently occurs among Indonesian students in examinations. Significance –This study provides further substantiation in the field of scientific reasoning and cognitive science; that a trend of confidence accuracy change in scientific reasoning test has been observed. It also contributes to uncovering the true ability of Indonesian students when performing such reasoning tests through their repeated attempts.