Brain-wave (EEG) recognition using transformers, an emerging machine learning technique

• Main Author: He, Linyi
• Other Authors: Wang Lipo
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Computer science and engineering::Computer applications; Engineering::Electrical and electronic engineering::Control and instrumentation
• Online Access: https://hdl.handle.net/10356/169334
• Institution: Nanyang Technological University

Electroencephalograph (EEG) signals play an important role in many aspects of brain science research and applications, and in order to make EEG signals better used in the scientific research, certain algorithms or systems need to be developed for EEG signal recognition. In this dissertation, I try to reproduce and improve two existing state-of-art transformer-based neural network methods of the EEG signal recognition. At present, the recognition of EEG signals mainly relies on convolutional neural networks or recurrent neural networks, but the structure of CNN is relatively fixed, and due to their structure and the characteristics of feature extraction, both CNN and RNN are unable to perceive the global features other than local features. EEG signal, as a signal with global features with global dependencies, new algorithm or system structure is needed. The first method I used in dissertation is Spatial Temporal Tiny Transformer (S3T) from [1]. It is a deep neural network with transformer-based structure. In this model, after some pre-processing of the native EEG signal, the Transformer built in S3T based on the attention mechanism performs attention-related processing on the spatial features of the data, followed by attention processing on the temporal features. Finally, the model slice the data, and then fully-connected layer performs the signal recognition. Although S3T model is powerful in EEG recognition, but for an pure attention mechanism model, S3T can only perceive and obtain the global features of EEG signals mainly. To some extent, it cannot perceive the local temporal features of EEG signals well. In order to solve the problem of obtaining local features and global features simultaneously, I try to use Convolutional Transformer (EEG Conformer). EEG Conformer is a hybrid network of convolutional neural network and Transformer designed by Song and his colleagues in [2]. In this model, first the convolutional part of the model will extract the temporal features as well as spatial features in the local range of EEG signals in a one-dimensional space, respectively. After that, Transformer structure learns the global dependencies on these low level structures. Finally, the fully-connected layer perform signal recognition on these extracted features. Also, I try to do some modification on the model structure or the hyperparameters. The purposes of modification includes: First, reduce the parameter size of the model and keep the recognition correct rate. Second, modify the hyper-parameters in the model to get a better recognition result. However, I have failed on these tries. For the first purpose, I do some ablation and combination work on the model, but did not keep the recognition rate at a satisfied level. The reasons of fail in this part are mainly because the combination or ablation lose important feature extraction ability in the model and result in great drop in recognition rate. For the second purpose, the modification on the hyperparameters only results in the drop of the recognition rate in the both models. The existing model have the parameters adjusted to a saturated level, the rise of them does not improve the recognition ability of the model significantly, but the reduction of them result in great loss in recognition correct rate. In this dissertation, I tried to use two public dataset to verify the recognition ability of the two models, and the results shows that the reproduced models are both acquire competitive recognition ability compared to the existing models which are dominated by the CNN or RNN architectures in EEG signal recognition nowadays. Keywords: EEG recognition; Transformer; attention mechanism; Convolutional Transformer(Conformer); Spatial-Temporal Tiny Transformer(S3T) ; neural network(nn); deep learning; motor imagery(MI).