A transformer framework for generating context-aware knowledge graph paths

• Main Authors: LO, Pei-Chi, LIM, Ee-peng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Knowledge graph; Contextual path generation; Information retrieval; Language model; Databases and Information Systems; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/8026
• Institution: Singapore Management University

Contextual Path Generation (CPG) refers to the task of generating knowledge path(s) between a pair of entities mentioned in an input textual context to determine the semantic connection between them. Such knowledge paths, also called contextual paths, can be very useful in many advanced information retrieval applications. Nevertheless, CPG involves several technical challenges, namely, sparse and noisy input context, missing relations in knowledge graphs, and generation of ill-formed and irrelevant knowledge paths. In this paper, we propose a transformer-based model architecture. In this approach, we leverage a mixture of pre-trained word and knowledge graph embeddings to encode the semantics of input context, a transformer decoder to perform path generation controlled by encoded input context and head entity to stay relevant to the context, and scaling methods to sample a well-formed path. We evaluate our proposed CPG models derived using the above architecture on two real datasets, both consisting of Wikinews articles as input context documents and ground truth contextual paths, as well as a large synthetic dataset to conduct larger-scale experiments. Our experiments show that our proposed models outperform the baseline models, and the scaling methods contribute to better quality contextual paths. We further analyze how CPG accuracy can be affected by different amount of context data, and missing relations in the knowledge graph. Finally, we demonstrate that an answer model for knowledge graph questions adapted for CPG could not perform well due to the lack of an effective path generation module.