EEG-based mental states recognition for arousal self-regulation system

• Main Author: Ee, Winson Yong Wei
• Other Authors: Guan Cuntai
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/156451
• Institution: Nanyang Technological University

Stress and anxiety are prevalent in our daily lives, which tend to negatively impact the performance of daily tasks. Conventional non-medicinal remedies include behavioral self-regulation of heightened arousal states. Though such behavioral interventions are ideal and viable, lack of adherence to daily practices and misjudgment due to subjective assessment could render such intervention ineffective. The requirement for subjective assessment by specialists causes such interventions to be costly and inefficient for individual patients. By leveraging Brain Computer Interface (BCI), EEG-based mental states recognition is actualized in various applications. Passive BCI technologies are extended to consumer markets to help improve our daily lives beyond initial clinical and research settings. Nevertheless, issues exist in BCI that limit its potential in day-to-day scenarios such as inter-subject and intra-subject variability. Diverse user responses and limited effectiveness of experiment stimuli bring about additional challenges in achieving robust arousal levels recognition. Currently, the calibration tasks for pro- Vs anti- arousal states use the same audio-visual stimuli across all subjects, resulting in poor elicitation of desirable mental states. These poorly representative EEG signals lead to suboptimal classifier models, resulting in unreliable performance. Furthermore, most experiments do not consider the subject’s baseline mental states such as attention, affects and working memory. These baseline mental states could potentially influence arousal states in model calibration tasks. Therefore, the main goal of this report is to design and develop an EEG-based mental state recognition system for arousal self-regulation that leverages BCI, neurofeedback, deep learning, and virtual reality (VR) technologies. This report proposes an implementation of a two-stage calibration paradigms that incorporates the use of personalized stimuli based on the subject’s preferences. Multimodal data is then collected from 35 healthy subjects according to the proposed experiment protocols. With the collected EEG data, the classification performance of pro Vs anti arousal states using subject-independent cross-validation schemes with a CNN-based deep learning approach are evaluated. A Python-based VR training application is then designed and implemented for real-time Arousal Management training that interfaces with an EEG-based arousal scoring system. I evaluated accuracy performance using pre-processed band-pass filtered EEG data with CNN-based deeper TSception models in leave-one-subject-out subject independent cross validations. I found the maximum mean accuracy of 88.10±12.7% for 4 seconds segment with 95% overlapped in a pair of pro Vs anti arousal task. I also identified that displaying arousal video with personalized stimuli yielded only 61.07±11.19% mean accuracy. Although we achieved high subject-independent arousal classification accuracy with specific pro Vs anti-arousal task pair, the advantages of personalized stimuli cannot be proven yet. I believe further performance evaluation and statistical analysis might shed lights on identifying the possible insights of how baseline tasks influence the arousal task as well as how different personalized stimuli affect the performance outcomes. Furthermore, the integration of VR-based Arousal Management system with online EEG arousal scores will provide a complete platform to evaluate the real-time performance of user’s arousal states.