Debiasing NLU models via causal intervention and counterfactual reasoning

• Main Authors: TIAN, Bing, CAO, Yixin, ZHANG, Yong, XING, Chunxiao
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Artificial Intelligence and Robotics; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/7454; https://ink.library.smu.edu.sg/context/sis_research/article/8457/viewcontent/21389_Article_Text_25402_1_2_20220628.pdf
• Institution: Singapore Management University

Recent studies have shown that strong Natural Language Understanding (NLU) models are prone to relying on annotation biases of the datasets as a shortcut, which goes against the underlying mechanisms of the task of interest. To reduce such biases, several recent works introduce debiasing methods to regularize the training process of targeted NLU models. In this paper, we provide a new perspective with causal inference to fnd out the bias. On the one hand, we show that there is an unobserved confounder for the natural language utterances and their respective classes, leading to spurious correlations from training data. To remove such confounder, the backdoor adjustment with causal intervention is utilized to fnd the true causal effect, which makes the training process fundamentally different from the traditional likelihood estimation. On the other hand, in inference process, we formulate the bias as the direct causal effect and remove it by pursuing the indirect causal effect with counterfactual reasoning. We conduct experiments on large-scale natural language inference and fact verifcation benchmarks, evaluating on bias sensitive datasets that are specifcally designed to assess the robustness of models against known biases in the training data. Experimental results show that our proposed debiasing framework outperforms previous stateof-the-art debiasing methods while maintaining the original in-distribution performance.