Fault detection method based on high-level feedback and reconstruction for image recognition

• Main Author: Xu, Jin
• Other Authors: Xie Ming
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision; Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/146731
• Institution: Nanyang Technological University

Image recognition problem is a typical challenge for artificial intelligence. Deep learning methods have shown great progress in this area in recent years. However, this approach has the limitation of outputting wrong result with high confidence when the input is not well represented in the training data, such as adversarial examples (natural images with intentionally designed small perturbations that are imperceptible by human beings) and unusual images that is not covered in the training. This limitation causes great safety concerns for the deployment of existing deep learning systems for critical applications such as autonomous driving and medical diagnosis. However, it is not practical to include all adversarial or unusual images in training because images are highly variable. It is therefore desirable to design an image recognition system with the ability of fault detection. When an “unseen” image, either adversarial or unusual, is input to the system, the fault detection mechanism recognizes the situation and raise a red flag. With the fault detected, further steps such as intervention like double checking by a person can be taken to avoid damages. Inspired by the human perception system, the research reported in this thesis focusses on a novel fault detection mechanism and a reconstruction process. This mechanism, named Detection Based on Feedback and Reconstruction (DBFR), requires no change of the existing recognition system and generates a reconstructed image as an intuitive “internal model” to indicate whether the interpretation is supported by what is learned in the training while performing the inference. By comparing the reconstructed image with the input image, the incorrect interpretation can be detected with high probability. Experiments validate the superior effectiveness of this method over existing defense methods and the reconstructed images help to interpret the internal decision process of the deep learning system, which is currently treated as a black box in most applications.