Neural network-based echocardiogram video classification by incorporating dynamic information and attention mechanism

• Main Author: Ye, Zi
• Format: Thesis
• Language: English; English
• Published: 2022
• Subjects: Q Science (General); QA Mathematics
• Online Access: http://eprints.utem.edu.my/id/eprint/26988/1/Neural%20network-based%20echocardiogram%20video%20classification%20by%20incorporating%20dynamic%20information%20and%20attention%20mechanism.pdf; http://eprints.utem.edu.my/id/eprint/26988/2/Neural%20network-based%20echocardiogram%20video%20classification%20by%20incorporating%20dynamic%20information%20and%20attention%20mechanism.pdf; http://eprints.utem.edu.my/id/eprint/26988/; https://plh.utem.edu.my/cgi-bin/koha/opac-detail.pl?biblionumber=122132
• Institution: Universiti Teknikal Malaysia Melaka

Echocardiography, the use of ultrasound waves to investigate the heart's action, is the primary physiological test for cardiovascular disease diagnoses. The determination of the probe viewpoint forms an essential step in echocardiographic image analysis. Some of such views are identified as standard views due to the presentation and ease of the evaluations of their major cardiac structures. Finding valid cardiac views has traditionally been a laborious process and interpreted manually by the specialist, so there exists significant interest in providing an automated classification of echocardiograms in order to speed up the diagnosis process. However, the traditional machine learning methods require time-consuming and operator-dependent manual selection and annotation of features. Therefore, this study aims to simplify the diagnosis process by providing an automated classification of standard cardiac views based on deep learning technology. More importantly, our research considers and assesses some new neural network architectures driven by action recognition in video. For this aim, two classes of neural network architectures have been outlined: the CNN+BiLSTM model and the Spatiotemporal-BiLSTM model. It is finally verified that these methods aggregating dynamic information receive a more substantial classification effect. In addition, previous observations concluded that the most significant challenge Hes in distingnishing among the various adjacent views. To this end, our study further aimed to adopt the attention mechanism for designing efficient neural networks. We proposed an ECHOAttention architecture consisting of two parts. We first design an ECHO-ACTION block, which efficiently encodes Spatio-temporal features, channel-wise features, and motion features. Then, we insert this block into existing ResNet architectures, combined with a self-attention module to ensure its echocardiogram classification task-related focus, to form an effective ECHO-Attention network. All of these experiments are implemented on our privately collected dataset of 2693 videos acquired from 267 patients, which trained cardiologists have manually labeled. The evidence from this study showed that all the proposed methods yielded good results. The ECHO-Attention architecture provides the best classification performance (overall accuracy of 94.81 % ) on the entire video sample and achieved significant improvements on the classification of anatomically similar views (precision 88.65% and 81.70% for PSAX-AP and PSAX-MID on 30-frame clips, respectively).