Unsupervised deep thermal RGB fusion for home applications with PyTorch framework

• Main Author: Liu, Yan
• Other Authors: Y. C. Chen
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Computer science and engineering; Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/140390
• Institution: Nanyang Technological University

With the development of the artificial intelligence, smart home has been introduced to our daily life. Under this thesis, thermal sensors are widely used for various applications. A low-resolution (LR) thermal sensor can only detect the object’s temperature while a high-resolution (HR) one can show how the temperature spreads on the object. However, the price and the feasibility of the thermal sensors are quite resolution dependent. A good thermal sensor that can monitor the temperature distribution are usually at high costs, which makes it less worthy to be applied in the smart home applications. This project proposes an unsupervised cross-domain guided image super-resolution method to enlarge the thermal resolution with the guided of HR RGB images. We also propose a new loss function to learn from the content of the LR thermal image and preserve the edge details from the high resolution RGB image. We conduct experiment and the results show that the proposed method produces good HR images with comparable quality as the costly HR thermal devices. A prototype is also built to demonstrate its performance.