Generating adversarial examples with only one image

• Main Author: Luo, Jinqi
• Other Authors: Jun Zhao
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/148573

Deep learning based vision systems are widely deployed in today's world. The backbones of these systems, namely deep neural networks (DNNs), are showing an impressive capability on feature extraction, large-scale training, and precise predictions. However, DNNs have been shown vulnerable to adversarial examples of different types including adversarial perturbations and adversarial patches. Existing approaches for adversarial patch generation hardly consider the contextual consistency between patches and the image background, causing such patches to be easily detected and adversarial attacks to fail. Additionally, these methods require a large amount of data for training, which is computationally expensive and time-consuming. In this project, we explore how to generate advanced adversarial patches effectively and efficiently. To overcome the aforementioned challenges, we propose an approach to generate adversarial yet inconspicuous patches with one single image. In our approach, adversarial patches are produced in a coarse-to-fine way with multiple scales of generators and discriminators. We consider the perceptual sensitivity of victim model by highlighting its sensitivity to equip our approach with strong attacking capability. The selection of patch location is based on the perceptual sensitivity of victim models. Contextual information is encoded during the Min-Max training to make patches consistent with surroundings. Through extensive experiments, our approach shows strong attacking ability in both the white-box and black-box setting. Experiments on saliency detection and user evaluation indicate that our adversarial patches, which can evade human observations, are more inconspicuous and natural-looking compared to existed approaches. Lastly, the experiments on real-world objects shows that our digital approach has the potential of being malicious in real-world settings.