DESIGN AND EVALUATION OF ASSESSMENT IN SIMPRAG (SIMULASI PRAKTIKUM GAME) USING RELIABILITY AND VALIDITY INDICATORS

• Main Author: Aprilliawati Putri S., Tita
• Format: Theses
• Language: Indonesia
• Online Access: https://digilib.itb.ac.id/gdl/view/58030
• Institution: Institut Teknologi Bandung

Since March of 2020, Coronavirus Disease 19 (COVID-19) pandemic has been affecting the learning process at school. Vocational High Schools that use a general rule in dividing the theory learning and practicing, 25% and 75%, respectively, could not fulfill the ratio since students have been studying by themselves lately. Therefore, one method to maximize the learning process between teachers and students is by utilizing internet technology to do online learning. Hence, the author designed a direct-connected assessment to a game named SIMPRAG (Simulasi Praktikum Game - Game Practice Simulation) to help teachers and students in online practicing. Although the process is virtual, the author wished that the assessment in SIMPRAG Game could push the students' will to learn and also expected this to help teachers in monitoring their students' learning process from afar. SIMPRAG focuses on Electric Lighting Installation in Vocational High School Electronics Major. There were three assessment processes in SIMPRAG Game: Pre-Game, Experiment Game, and Post-Game, representing pre-test, experiment, and post-test. The author used an alternative assessment method in both Experiment Game and Post-Game, eliminating the time limit, whereas the author used a 15-minute time limit in Pre-Game. Every student could play the SIMPRAG Game by downloading the application from E-SIMPRAG (Electronic Simulasi Praktikum Game) website and install it to their device. These two applications give easy access to material and job sheets to students for practical use. It also provides practice results monitoring to teachers. By integrating these two applications, such as game login, user data checking, and practice results data sending from game to website via database, the author could provide the easiness mentioned before. The author used a modular system provided by Software Unity Engine named UnityWebRequest to integrate the system. ADDIE (Analysis, Design, Development and Implementation, and Evaluation) is used as research method and the heuristic method to grab two elements (education and content) is used to evaluate questionnaire results. An assessment is categorized as a good one when it could fulfill two crucial indicators: reliability and validity. The author proved these two indicators by evaluating 18 students and two teachers as the experts at SMKN 1 Katapang, Bandung, West Java, and SMK Auto Matsuda, Kuningan, West Java. The evaluation consisted of the game trial and questionnaire. The author used the test-retest reliability method to prove that the assessment was stable and could show students' achievement based on their skills. It was executed by comparing two scores: Job 1 assessment total duration and Job 2 assessment total duration. Those two scores are directly proportional in scatter plot display, meaning that the students who did Job 1 in a relatively short time also did Job 2 in the same manner, vice versa. T-test method also shows the result of 0.38 that has a greater value than the standard p-value of 0.05. Thus, showing that this assessment was reliable. Besides, the author used the content validity method is to prove the validity that needs affirmatory from an expert, reassuring that the evaluation is indeed in line with the learning syllabus and material. Questionnaire results show that the average score is 85% from students and 88.33% from teachers in the content element. Therefore, the author could confirm that the SIMPRAG Game assessment has good validity. Aside from that two important indicators, the author also took another score in the questionnaire, including education, application use, and the communication easiness from system integration. This experiment got an average of 86% from students and 88.33% from teachers in the education element, showing that the SIMPRAG Game assessment is a decent learning tool. Teachers and students also give a 90% score on how much they felt being helped by SIMPRAG Game and E-SIMPRAG Website. Thus, this could be a scoring alternative to online practical experiments. Students and teachers also give 89% and 100% scores, respectively, in the application use element. It shows that they could log in easily in SIMPRAG Game. Lastly, students and teachers give 83% and 100% scores, respectively, in the communication easiness from the system integration element. It demonstrates that sending practice results from the SIMPRAG Game to the ESIMPRAG Website is a simple process.