CoVariance-based causal debiasing for entity and relation extraction

• Main Authors: REN, Lin, LIU, Yongbin, CAO, Yixin, OUYANG, Chunping
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/8395; https://ink.library.smu.edu.sg/context/sis_research/article/9398/viewcontent/2023.findings_emnlp.173.pdf
• Institution: Singapore Management University

Joint entity and relation extraction tasks aim to recognize named entities and extract relations simultaneously. Suffering from a variety of data biases, such as data selection bias, and distribution bias (out of distribution, long-tail distribution), serious concerns can be witnessed to threaten the model’s transferability, robustness, and generalization. In this work, we address the above problems from a causality perspective. We propose a novel causal framework called covariance and variance optimization framework (OVO) to optimize feature representations and conduct general debiasing. In particular, the proposed covariance optimizing (COP) minimizes characterizing features’ covariance for alleviating the selection and distribution bias and enhances feature representation in the feature space. Furthermore, based on the causal backdoor adjustment, we propose variance optimizing (VOP) separates samples in terms of label information and minimizes the variance of each dimension in the feature vectors of the same class label for mitigating the distribution bias further. By applying it to three strong baselines in two widely used datasets, the results demonstrate the effectiveness and generalization of OVO for joint entity and relation extraction tasks. Furthermore, a fine-grained analysis reveals that OVO possesses the capability to mitigate the impact of long-tail distribution. The code is available at https://github.com/HomuraT/OVO.