Brain-wave (EEG) recognition using transformers, an emerging machine learning technique

Electroencephalograph (EEG) signals play an important role in many aspects of brain science research and applications, and in order to make EEG signals better used in the scientific research, certain algorithms or systems need to be developed for EEG signal recognition. In this dissertation, I try t...

Full description

Saved in:
Bibliographic Details
Main Author: He, Linyi
Other Authors: Wang Lipo
Format: Thesis-Master by Coursework
Language:English
Published: Nanyang Technological University 2023
Subjects:
Online Access:https://hdl.handle.net/10356/169334
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-169334
record_format dspace
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering::Computer applications
Engineering::Electrical and electronic engineering::Control and instrumentation
spellingShingle Engineering::Computer science and engineering::Computer applications
Engineering::Electrical and electronic engineering::Control and instrumentation
He, Linyi
Brain-wave (EEG) recognition using transformers, an emerging machine learning technique
description Electroencephalograph (EEG) signals play an important role in many aspects of brain science research and applications, and in order to make EEG signals better used in the scientific research, certain algorithms or systems need to be developed for EEG signal recognition. In this dissertation, I try to reproduce and improve two existing state-of-art transformer-based neural network methods of the EEG signal recognition. At present, the recognition of EEG signals mainly relies on convolutional neural networks or recurrent neural networks, but the structure of CNN is relatively fixed, and due to their structure and the characteristics of feature extraction, both CNN and RNN are unable to perceive the global features other than local features. EEG signal, as a signal with global features with global dependencies, new algorithm or system structure is needed. The first method I used in dissertation is Spatial Temporal Tiny Transformer (S3T) from [1]. It is a deep neural network with transformer-based structure. In this model, after some pre-processing of the native EEG signal, the Transformer built in S3T based on the attention mechanism performs attention-related processing on the spatial features of the data, followed by attention processing on the temporal features. Finally, the model slice the data, and then fully-connected layer performs the signal recognition. Although S3T model is powerful in EEG recognition, but for an pure attention mechanism model, S3T can only perceive and obtain the global features of EEG signals mainly. To some extent, it cannot perceive the local temporal features of EEG signals well. In order to solve the problem of obtaining local features and global features simultaneously, I try to use Convolutional Transformer (EEG Conformer). EEG Conformer is a hybrid network of convolutional neural network and Transformer designed by Song and his colleagues in [2]. In this model, first the convolutional part of the model will extract the temporal features as well as spatial features in the local range of EEG signals in a one-dimensional space, respectively. After that, Transformer structure learns the global dependencies on these low level structures. Finally, the fully-connected layer perform signal recognition on these extracted features. Also, I try to do some modification on the model structure or the hyperparameters. The purposes of modification includes: First, reduce the parameter size of the model and keep the recognition correct rate. Second, modify the hyper-parameters in the model to get a better recognition result. However, I have failed on these tries. For the first purpose, I do some ablation and combination work on the model, but did not keep the recognition rate at a satisfied level. The reasons of fail in this part are mainly because the combination or ablation lose important feature extraction ability in the model and result in great drop in recognition rate. For the second purpose, the modification on the hyperparameters only results in the drop of the recognition rate in the both models. The existing model have the parameters adjusted to a saturated level, the rise of them does not improve the recognition ability of the model significantly, but the reduction of them result in great loss in recognition correct rate. In this dissertation, I tried to use two public dataset to verify the recognition ability of the two models, and the results shows that the reproduced models are both acquire competitive recognition ability compared to the existing models which are dominated by the CNN or RNN architectures in EEG signal recognition nowadays. Keywords: EEG recognition; Transformer; attention mechanism; Convolutional Transformer(Conformer); Spatial-Temporal Tiny Transformer(S3T) ; neural network(nn); deep learning; motor imagery(MI).
author2 Wang Lipo
author_facet Wang Lipo
He, Linyi
format Thesis-Master by Coursework
author He, Linyi
author_sort He, Linyi
title Brain-wave (EEG) recognition using transformers, an emerging machine learning technique
title_short Brain-wave (EEG) recognition using transformers, an emerging machine learning technique
title_full Brain-wave (EEG) recognition using transformers, an emerging machine learning technique
title_fullStr Brain-wave (EEG) recognition using transformers, an emerging machine learning technique
title_full_unstemmed Brain-wave (EEG) recognition using transformers, an emerging machine learning technique
title_sort brain-wave (eeg) recognition using transformers, an emerging machine learning technique
publisher Nanyang Technological University
publishDate 2023
url https://hdl.handle.net/10356/169334
_version_ 1772825169185734656
spelling sg-ntu-dr.10356-1693342023-07-14T15:43:23Z Brain-wave (EEG) recognition using transformers, an emerging machine learning technique He, Linyi Wang Lipo School of Electrical and Electronic Engineering ELPWang@ntu.edu.sg Engineering::Computer science and engineering::Computer applications Engineering::Electrical and electronic engineering::Control and instrumentation Electroencephalograph (EEG) signals play an important role in many aspects of brain science research and applications, and in order to make EEG signals better used in the scientific research, certain algorithms or systems need to be developed for EEG signal recognition. In this dissertation, I try to reproduce and improve two existing state-of-art transformer-based neural network methods of the EEG signal recognition. At present, the recognition of EEG signals mainly relies on convolutional neural networks or recurrent neural networks, but the structure of CNN is relatively fixed, and due to their structure and the characteristics of feature extraction, both CNN and RNN are unable to perceive the global features other than local features. EEG signal, as a signal with global features with global dependencies, new algorithm or system structure is needed. The first method I used in dissertation is Spatial Temporal Tiny Transformer (S3T) from [1]. It is a deep neural network with transformer-based structure. In this model, after some pre-processing of the native EEG signal, the Transformer built in S3T based on the attention mechanism performs attention-related processing on the spatial features of the data, followed by attention processing on the temporal features. Finally, the model slice the data, and then fully-connected layer performs the signal recognition. Although S3T model is powerful in EEG recognition, but for an pure attention mechanism model, S3T can only perceive and obtain the global features of EEG signals mainly. To some extent, it cannot perceive the local temporal features of EEG signals well. In order to solve the problem of obtaining local features and global features simultaneously, I try to use Convolutional Transformer (EEG Conformer). EEG Conformer is a hybrid network of convolutional neural network and Transformer designed by Song and his colleagues in [2]. In this model, first the convolutional part of the model will extract the temporal features as well as spatial features in the local range of EEG signals in a one-dimensional space, respectively. After that, Transformer structure learns the global dependencies on these low level structures. Finally, the fully-connected layer perform signal recognition on these extracted features. Also, I try to do some modification on the model structure or the hyperparameters. The purposes of modification includes: First, reduce the parameter size of the model and keep the recognition correct rate. Second, modify the hyper-parameters in the model to get a better recognition result. However, I have failed on these tries. For the first purpose, I do some ablation and combination work on the model, but did not keep the recognition rate at a satisfied level. The reasons of fail in this part are mainly because the combination or ablation lose important feature extraction ability in the model and result in great drop in recognition rate. For the second purpose, the modification on the hyperparameters only results in the drop of the recognition rate in the both models. The existing model have the parameters adjusted to a saturated level, the rise of them does not improve the recognition ability of the model significantly, but the reduction of them result in great loss in recognition correct rate. In this dissertation, I tried to use two public dataset to verify the recognition ability of the two models, and the results shows that the reproduced models are both acquire competitive recognition ability compared to the existing models which are dominated by the CNN or RNN architectures in EEG signal recognition nowadays. Keywords: EEG recognition; Transformer; attention mechanism; Convolutional Transformer(Conformer); Spatial-Temporal Tiny Transformer(S3T) ; neural network(nn); deep learning; motor imagery(MI). Master of Science (Computer Control and Automation) 2023-07-13T05:14:04Z 2023-07-13T05:14:04Z 2023 Thesis-Master by Coursework He, L. (2023). Brain-wave (EEG) recognition using transformers, an emerging machine learning technique. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/169334 https://hdl.handle.net/10356/169334 en application/pdf Nanyang Technological University