Adversarial attacks and defenses in natural language processing

• Main Author: Dong, Xinshuai
• Other Authors: Luu Anh Tuan
• Format: Thesis-Master by Research
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/159029
• Institution: Nanyang Technological University

Deep neural networks (DNNs) are becoming increasingly successful in many fields. However, DNNs are shown to be strikingly susceptible to adversarial examples. For instance, models pre-trained on very large corpora can still be easily fooled by word substitution attacks using only synonyms. This phenomenon has raised grand security challenges to modern machine learning systems, such as self-driving, spam filtering, and speech recognition, where DNNs are widely deployed. In this thesis, we first give a brief introduction of adversarial attacks and defenses. We focus on Natural Language Processing (NLP) and review some recent advances in attack algorithms and defense methods in Chapter 2. We also give a formalized definition of the research objective in this thesis, i.e., how to improve the adversarial robustness of NLP models. To this end, we propose novel and effective solutions to enhance NLP models towards robustness in the following chapters. In Chapter 3, for the classical NLP models like Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN), we present a novel adversarial training method, Adversarial Sparse Convex Combination (ASCC) defense, for adversarial robustness against word substitution attacks. To be specific, we model the substitution attack space as a convex hull and employ a regularizer to encourage the modeled perturbation towards an actual substitution. Therefore, we are able to align the modeling better with the discrete textual space. We empirically validate ASCC-defense in our experiments and it surpasses all compared state-of-the-arts on prevailing NLP tasks like sentiment analysis and natural language inference consistently under multiple attacks. To date, pre-trained language models, e.g., Bidirectional Transformers (BERT), are getting increasingly popular and fine-tuning a pre-trained language model for downstream tasks is becoming the new NLP paradigm. As such, how to fine-tune pre-trained language models towards adversarial robustness is of great importance. In Chapter 4, we first demonstrate that the prevalent defense technique, adversarial training, does not directly fit a conventional fine-tuning scenario. The reason lies in that conventional adversarial fine-tuning suffers severely from catastrophic forgetting and the fine-tuned models often fail to retain the generic and robust linguistic features captured during the pre-training stage. To this end, we propose Robust Informative Fine-Tuning (RIFT), a novel adversarial fine-tuning method from an information-theoretical perspective. In particular, RIFT encourages a model to memorize all the useful features learned before throughout the entire fine-tuning process, whereas a conventional fine-tuning framework only uses the weights of the pre-trained model for initialization. In experiments, we demonstrate that RIFT consistently surpasses state-of-the-arts under different attacks across various pre-trained language models. Last, we conclude this thesis in Chapter 5 and discuss some promising future directions for further exploration.