Attention multihop graph and multiscale convolutional fusion network for hyperspectral image classification

• Main Authors: Zhou, Hao, Luo, Fulin, Zhuang, Huiping, Weng, Zhenyu, Gong, Xiuwen, Lin, Zhiping
• Other Authors: School of Electrical and Electronic Engineering
• Format: Article
• Language: English
• Published: 2023
• Subjects: Engineering::Electrical and electronic engineering; Attention Fusion; Convolutional Neural Network
• Online Access: https://hdl.handle.net/10356/172259
• Institution: Nanyang Technological University

Convolutional neural networks (CNNs) for hyperspectral image (HSI) classification have generated good progress. Meanwhile, graph convolutional networks (GCNs) have also attracted considerable attention by using unlabeled data, broadly and explicitly exploiting correlations between adjacent parcels. However, the CNN with a fixed square convolution kernel is not flexible enough to deal with irregular patterns, while the GCN using the superpixel to reduce the number of nodes will lose the pixel-level features, and the features from the two networks are always partial. In this article, to make good use of the advantages of CNN and GCN, we propose a novel multiple feature fusion model termed attention multihop graph and multiscale convolutional fusion network (AMGCFN), which includes two subnetworks of multiscale fully CNN and multihop GCN to extract the multilevel information of HSI. Specifically, the multiscale fully CNN aims to comprehensively capture pixel-level features with different kernel sizes, and a multihead attention fusion module (MAFM) is used to fuse the multiscale pixel-level features. The multihop GCN systematically aggregates the multihop contextual information by applying multihop graphs on different layers to transform the relationships between nodes, and an MAFM is adopted to combine the multihop features. Finally, we design a cross-attention fusion module (CAFM) to adaptively fuse the features of two subnetworks. The AMGCFN makes full use of multiscale convolution and multihop graph features, which is conducive to the learning of multilevel contextual semantic features. Experimental results on three benchmark HSI datasets show that the AMGCFN has a better performance than a few state-of-the-art methods. Code: https://github.com/EdwardHaoz/IEEE_TGRS_AMGCFN.