Spectral reflectance estimation from RGB images with deep learning

• Main Author: Liao, Xuanli
• Other Authors: Wen Bihan
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering; Multispectral image; Reflectance separation; Deep learning
• Online Access: https://hdl.handle.net/10356/181566
• Institution: Nanyang Technological University

Multispectral images capture detailed spectral information across multiple bands, making them essential for applications in remote sensing, agriculture, medical imaging, and industrial inspection. With multispectral data, precise object recog- nition, component analysis, and health monitoring become possible, significantly improving accuracy. Despite their advantages, the high expense and complexity of multispectral image equipment limited its practical use. As a promising alter- native, generating multispectral images from RGB data using machine learning has gained traction, mapping RGB values to multispectral bands. However, this approach often carries over lighting effects from the RGB image. In contrast, an illumination-corrected multispectral image isolates the object’s true spectral reflectance, independent of lighting conditions, and is better suited for applica- tions requiring intrinsic object properties, such as plant health monitoring and material identification. In this project, we employ a machine learning framework that estimates and removes illumination from RGB images, isolating object reflectance and generating multispectral data more accurately reflective of the object’s natural properties. Through matrix-based performance assessment, we compare multispec- tral reflectance generated with and without illumination correction. Results show that the illumination-corrected approach yields significantly improved GFC and RMSE values, especially for select camera models such as Canon 1D Mark III, highlighting its enhanced spectral reconstruction accuracy. This research demonstrates the feasibility of RGB-derived multispectral imaging for more accurate spectral estimation, making multispectral imaging more accessible and suitable for real-world applications that demand high fidelity in spectral data.