Semi-supervised generative open information extraction

• Main Author: Zhou, Shaowen
• Other Authors: Long Cheng
• Format: Thesis-Master by Research
• Language: English
• Published: Nanyang Technological University 2023
• Subjects: Engineering::Computer science and engineering
• Online Access: https://hdl.handle.net/10356/171930
• Institution: Nanyang Technological University

Open Information Extraction (OpenIE) extracts facts in the form of n-ary relation tuples, i.e., (arg1, predicate, arg2, …, argn), from unstructured text without relying on predefined ontology schema. It has the potential to handle heterogeneous corpora with minimal human intervention. With OpenIE, Web-scale unconstrained IE systems can be developed to acquire large quantities of knowledge. The knowledge gathered can then be integrated and used in various natural language processing (NLP) applications, such as textual entailment, summarization, question answering, and explicit reasoning. Firstly, we surveyed neural OpenIE. Thanks to the rapid development of deep learning technologies, numerous neural OpenIE architectures have been proposed, achieving considerable performance improvement. In this survey, we provide an extensive overview of the state-of-the-art neural OpenIE models, their key design decisions, strengths, and weaknesses. Then, we discuss the limitations of current solutions and the open issues in the OpenIE problem itself. Finally, we list recent trends that could help expand its scope and applicability, setting up promising directions for future research in OpenIE. This paper is the first review of neural OpenIE. Secondly, we consider semi-supervised generative OpenIE to boost the performance by utilizing unlabeled texts. Deep learning-based OpenIE systems need more high-quality training data. Moreover, as mentioned in previous work, different OpenIE applications focus on different types of information. Existing unsupervised data bootstrapping solutions need help producing high-quality training data and being more flexible in adjusting the data to be suitable for specific applications. Recently, generative methods have shown great potential in various NLP tasks. We leverage a mean teacher-based semi-supervised learning method to boost the performance of a generative OpenIE model. We first pre-train the model using existing training data and then conduct semi-supervised training using gold data and unlabeled corpus. To filter out non-trustworthy training examples generated during the semi-supervised learning phase, we introduce a verification mechanism to improve the quality of the examples. Instead of training two separate models, we use prompts to train a joint model with extraction and verification capabilities. We conduct experiments on OpenIE benchmarks and show the promising results achieved by our proposed method. Finally, we conduct a few experiments evaluating ChatGPT’s zero-shot capability, discuss possible future work directions, and conclude the report.