Deep neural networks for identifying causal relations in texts

• Main Author: Chen, Siyuan
• Other Authors: Mao Kezhi
• Format: Thesis-Master by Research
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/171180
• Institution: Nanyang Technological University

Causal relation identification is an important task that facilitates many downstream tasks such as why-question answering, event prediction, information retrieval, sentiment analysis, etc. The goal of causal relation identification is to determine whether there's causal relation between 2 events or entities. With the prosperity of deep learning, end-to-end deep neural models have made great progress. Such approaches mainly focus on learning causal patterns from the training data, i.e. data-oriented. However, causal relation identification is a highly difficult task due to the flexible linguistic expressions and the background knowledge needed for identification. Despite of the necessity of learning from training data, only relying on it may suffer from poor generalizability. Hence, it's important to incorporate external knowledge into the neural network to improve its generalizability. In this thesis, we explore this idea in 2 subfields of causal relation identification, namely emotion-cause pair extraction and event causality identification. Emotion-Cause Pair Extraction (ECPE) aims to extract emotion and cause clauses underlying a text and pair them. Most of the recent approaches to this problem adopt deep neural networks to learn the inter-clause dependencies from training data, without making full use of information of all granularities and external knowledge. In this thesis, we propose a model that utilizes external knowledge and multi-granular information, including word-level, clause-level, and document-level information, to facilitate emotion-cause pair extraction. More specifically, at clause-level, two fully-connected clause graphs are constructed, namely emotion graph and cause graph. Graph attention is applied to learn emotion-specific and cause-specific representations which are then used to generate document-level representations. To further exploit the mutual indication between emotion and cause clauses, a cross-graph co-attention mechanism is proposed. At word-level, external knowledge of emotional and causal cues is incorporated to provide word-level indicative information for emotion-cause pair extraction. The proposed model is tested on both Chinese [1] and English [2] datasets, and the results show that our model achieves state-of-the-art performance on both datasets. Event Causality Identification (ECI) aims at detecting the causal relation between 2 events, which is a challenging task due to the complexity of causal expressions and the background knowledge needed for identifying certain causal relations. Considerable work has been done on the learning of context and the incorporation of external knowledge. However, few of them combines the data-oriented model with both explicit and implicit causal knowledge. In this thesis, we propose an integrative model to integrate causal indicator words, implicit causal knowledge and data-oriented model to enhance causality identification. For the data-oriented model, an event pair graph is constructed and Relational Graph Convolutional Network (R-GCN) is employed to better capture interactions between individual pairs. For the causal indicators, their word embeddings are used to initialize the filters of convolutional neural network so as to capture the clues indicating causal relation. For the causal facts, we design a cause-effect matching mechanism to measure the possibility of causal relation holding between 2 events, based on the possible causes and effects automatically generated by COMET. The proposed method is evaluated on 3 datasets [3-5], and experimental results demonstrate the effectiveness of the proposed method.