System reliability enhancement via deep-driven computer vision

• Main Author: Ding, Shuya
• Other Authors: LUO Jun
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision
• Online Access: https://hdl.handle.net/10356/151578
• Institution: Nanyang Technological University

Recent developments in computer vision using deep learning techniques have caused broad-ranging in all kinds of applications. However, the use of deep-driven computer vision techniques in improving system reliability is still a niche research direction. In this thesis, we present research efforts via deep-driven computer vision techniques for en- hancing neural network based classification system reliability. Image Classification (IC) system and Radio-Frequency Based Human Activity Recognition (RF-HAR) system are selected as test-beds. The reliability enhancement is considered from two perspectives: i) enhance system robustness to defend against well-designed adversarial attacks, and ii) improve system performance in the face of extremely low data regime. Therefore, adversarial defense and knowledge transfer are the two main research objectives to be investigated in this thesis. To achieve these objectives, we consider the following ap- proaches: i) an attack-agnostic adversarial defense framework to enhance the intrinsic robustness of IC system when it is attacked by adversarial examples, ii) a meta-learning based approach to transfer knowledge from seen environments, leading to adaptation of RF-HAR system in new environment with data scarcity, and iii) a meta-learning based adversarial defense framework to transfer robustness from robust IC system, giving rise to non-robust IC system robustness enhancement even under data scarcity. To be specific, an attack-agnostic adversarial defense framework is first designed to improve the intrinsic robustness of IC system. Many researchers have proposed various adversarial attacks to generate adversarial examples. They can severely threaten the reliability of the system. Correspondingly, adversarial defense strategies are designed as countermeasures to improve system robustness. Although adversarial training is the most promising defense strategy by augmenting specific adversarial examples during training, it has limited classification system generalization capability and suffers from excessive time complexity. An attack-agnostic defense framework named Feature Pyramid Decoder (FPD) is proposed to enhance the intrinsic robustness of the classification system. Mean- while, it is without jeopardizing the ability to generalize clean samples. FPD could apply to all block-based systems. It integrates denoising layers and image restoration modules into a targeted system, and it also constrains the Lipschitz constant of the classification layer. Furthermore, a two-phase strategy is crafted for training the FPD-enhanced sys- tem. It utilizes -noisy neighborhood images with multi-task and self-supervised learning. Extensive experiments are conducted under a variety of well-known white-box attacks and black-box attacks on MNIST, the Street View House Numbers (SVHN), and CAL- TECH datasets. Results demonstrate that the FPD-enhanced system could improve intrinsic robustness against adversarial examples in an attack-agnostic manner. Besides, if adversarial training is conducted, the FPD-enhanced system performs better than their non-enhanced version. Afterward, we propose a meta-learning based approach to transfer knowledge from seen environments, giving rise to the capability of RF-HAR system being adaptive to new environments with very few labeled data. To be specific, RF-HAR system rises as a promising solution for many applications. However, RF sensing (device-free) is often more sensitive to environmental changes than device-based sensing. Also, RF datasets strictly require on-line labeling during collection, starkly different from image and text data collections where human interpretations can be leveraged to perform off- line labeling. Therefore, existing solutions to RF-HAR system entail a laborious data collection process for adapting to new environments. To this end, RF-Net as a unified meta-learning based knowledge transfer framework for one-shot RF-HAR is proposed; it reduces the labeling efforts for environment adaptation to the minimum level. We innovate in two designs: i) a dual-path base HAR network, where both time and frequency domains are dedicated to learning powerful RF features including spatial and attention- based temporal ones, and ii) a metric-based meta-learning framework learning from seen environments, where HAR network is enabled to transform input into a representation suitable for classification via the similarity comparison and then environment adaptation can be performed with very few labeled data. It includes an RF-specific metric module along with a residual classification module. We conduct extensive experiments on three representative RF sensing techniques, Wi-Fi radio, frequency-modulated continuous wave radio, and impulse radio, in multiple real-world indoor environments; all results strongly demonstrate the efficacy of RF-Net being adaptive to new environments with limited labeled data, compared with state-of-the-art baselines. Last but not least, we further employ knowledge transfer to transfer robustness in the form of robust features, from adversarially-trained robust (source) IC system to a non-robust (target) IC system, and consequently to enhance robustness of target IC system even in extremely low data regime. We design a meta-learning based adversarial defense framework to perform this few-shot robustness transfer. As mentioned above, adversarial training is deemed as a potent defense strategy against powerful adversarial attacks. However, it requires sufficient well-designed adversarial examples, leading to an extra burden on crafting adversarial examples. One recent trend to handle this issue focuses on direct transferring adversarial robustness from a robust system to a non-robust system. However, the performance of these existing solutions may decay severely in low data regimes. Therefore, it motivates us to propose Meta Robustness Transfer (MRT) empowered by a non-parametric Robust Feature Selection Layer (RFSL) as a meta- learning based adversarial defense framework for fighting the data scarcity in transferring robustness. Guided by a robust source system, MRT essentially enables an adapted target system to learn an embedding function; this function embeds samples into robust features without involving adversarial training, and it enables a robust sample classification via a simple similarity comparison, avoiding the need for extensive labelled data. Consequently, the robustness of the target system can be enhanced. Through experiments in Mini- ImageNet, Caltech-UCSD Birds, and CIFAR-FS, we show that our approach can transfer robustness across tasks with different system architectures, different label spaces, and different datasets, in extremely low data regimes.