Affect recognition using brainwaves and mouse behavior for math tutoring systems

• Main Authors: Ibañez, John Francis I., Jr., Lim, Ianne Robert C., Lumanas, Nestor B., Jr.
• Format: text
• Language: English
• Published: Animo Repository 2011
• Subjects: Intelligent tutoring systems; Mathematics > Study and teaching; Tutors and tutoring; Computer Sciences
• Online Access: https://animorepository.dlsu.edu.ph/etd_bachelors/10614
• Institution: De La Salle University

There are various researches focusing on emotions recognition which include using different modalities such as face and voice. Only few have studied brainwave as a mode for recognizing emotion. Brainwaves are difficult to mask therefore, this modality may provide a more accurate information on the affective state of the user. Another modality that has not been much explored is the standard input device, the mouse. Mouse behavior such as clicks and movements were correlated with a particular learning related affect. Moreover, personality traits may play a role in the affective experience of the user. Thus, this study aims to predict the intensity of academic related emotions of a person based on his/her brainwave signal, mouse behavior, context, and personality. This was accomplished by performing experiments from 25 volunteer with ages ranging from 17 to 23 years old. The subjects were asked to use a Math Tutoring System while an EEG sensor is attached to their head and used a standard input mouse. Mouse behavior data such as clicks, duration and distance travelled by the mouse were collected simultaneously with the EEG data. The participants were asked to self report their emotions during the session i.e confidence excitement, frustration and interest. The raw data were filtered and processed for feature extraction. Several feature selection and classification techniques were applied. The techniques yielding the highest accuracy were selected for building the final affect model for determining the level of confidence, frustration, excitement, and interest. Based on the results, the combination of beta and gamma frequency EEG bands combined with mouse data yielded the highest accuracy rate for frustration using C4.5 with an accuracy of 70.18% because these band are associated with active thinking activity. Four classifier were built for predicting the intensity of each emotion. Confidence was best classified using MLP and beta and gamma features with an accuracy of 67.35%. However, alpha bands without mouse feat