The impact of timer and warning alert in an online examination interface on the user experience of students integrated by openness to experience

• Main Authors: De Jose, Beatriz Anne T., Pasia, Danielle Alexandria A.
• Format: text
• Language: English
• Published: Animo Repository 2021
• Subjects: Examinations—Design and construction; Automatic timers; Educational tests and measurements; Industrial Engineering
• Online Access: https://animorepository.dlsu.edu.ph/etdb_induseng/7; https://animorepository.dlsu.edu.ph/context/etdb_induseng/article/1006/viewcontent/DeMesa2.pdf
• Institution: De La Salle University

As swift and significant growth are seen in online learning, this does not rule out the fact that exams have been a constant requirement in institutions, be it in person or online. Exams serve as an integral part in the academic lives of the students, which only makes it imperative for this study to look into the factors that make up the online examination interface facilitated by a commonly used platform: learning management systems (LMS). This study aims to create a set of guidelines which could aid in the process of developing online examination interfaces, which users may view as satisfying and useful. The effectiveness of each design factor’s configurations was tested on the users’ experience, specifically satisfaction and perceived usefulness, interactions effects were examined between these factors, and finally, the moderating effect of openness to experience was assessed on the relationship between the design factors and the user experiences. By integrating personality, this study was able to explore the effect of the design factors on their satisfaction and perceived usefulness and better understand their user experiences that allowed their effects to work or even fail.The configurations examined for timer and alert were countdown timer, rising counter, and traditional clock, and 5 minutes remaining, 15 minutes remaining, and mid-examination, respectively. These were selected based on the general preferences of various students from the literature without taking into account their user characteristics. A none configuration was also tested to determine if its presence was significant in itself. The participants were initially screened prior to the experiment proper as if they were taking an actual online test, which includes observing their location and position via Zoom, as well as ensuring a noise- and distraction- free environment. It was also important that they were willing to undergo the experiment, their perceived internet connection was enough to take online tests, and they were comfortable with the technology being used. Subsequently, an openness to experience personality test was given. The actual experiment involved an online exam that consisted of basic math and English skills questions which was good for 1 hour. The exam already had the timer and alert incorporated depending on the setting assigned to the participant. To cap the whole experiment, the participants ended with a post-survey that would collect their user experiences from the test. Overall, the user experience of 152 participants was observed and analyzed. From the results of the experiment, it was seen that for the user experiences, timer is significant to both satisfaction and perceived usefulness while alert only triggers perceived usefulness. On the other hand, there was no significant difference between the rising timer and traditional clock. For the results of the interaction analysis, it was found that an interaction between timer and alert only exists for one dependent variable, perceived usefulness. Specifically, an interaction is seen for rising counter and none when interacted 5 minutes remaining, 15 minutes remaining, and no alert. Moreover, the results proven in hypothesis 1 through the direct effect analysis show that satisfaction is only loaded on timer and not warning alert. A number of participants reported that when their respective warning alerts appeared in addition to the timers already displayed, it only pressured them more to answer the questions quickly, which diminished their performance. Openness to experience moderated the effect of alert to satisfaction and timer to perceived usefulness. In particular, the impact of openness to experience on the relationship between each warning alert to satisfaction only applies for the 15-minute and mid-examination warning alert. People with low openness to experience are less satisfied with the 15-minute alert but are highly satisfied with mid-exam alerts. On the other hand, moderation was found from the rising timer to perceived usefulness such that people who were open to experience found the rising timer to be less useful.