Hyperspectral reconstruction of SOTA method on natural image and remote sensing datasets

• Main Author: Zhou, Junzhe
• Other Authors: Wen Bihan
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering; Hyperspectral reconstruction; Cross-domain generalization
• Online Access: https://hdl.handle.net/10356/181539
• Institution: Nanyang Technological University

Hyperspectral image reconstruction has been widely used in remote sensing, agriculture, environmental monitoring and other fields. However, traditional reconstruction methods and CNN-based reconstruction methods have some limitations in accuracy and efficiency. In recent years, Transformer based models have shown excellent performance in hyperspectral reconstruction tasks, especially MST++ has demonstrated impressive spectral reconstruction capabilities on natural image datasets such as ARAD_1K, but its generalization capabilities in different scenes, especially remote sensing images, have not been comprehensively studied. In this dissertation, we systematically analyze and evaluate the spectral reconstruction performance of MST++ on different data sets. In this paper, we first review the existing methods of hyperspectral image reconstruction, and focus on the innovation of MST++. Then, the structure of MST++ model is explained in detail, including Spectral wise Multi-head Self-Attention (SMSA) and Feed Forward Network (FFN), and their roles in spectral reconstruction are analyzed. In the experimental part of this dissertation, MST++ is trained on natural image dataset (ARAD_1K) and remote sensing image dataset (AVIRIS) respectively, and then the spectral reconstruction performance of MST++ is tested on ARAD_1K, AVIRIS and Washington DC Mall respectively. The results show that MST++ has excellent performance in the codomain experiment, can accurately recover spectral information, and has excellent performance in structure preservation. However, in cross-domain experiments, the generalization ability of the model is decreased, especially when dealing with data sets with large spectral complexity and brightness differences, the model shows certain limitations. The results of this dissertation show the advantages and disadvantages of MST++, and point out possible future research directions, including but not limited to improving the cross-domain generalization ability of the model by introducing domain adaptive technology, and reducing its computational complexity by model lightweight technology.