VEM 2 L: An easy but effective framework for fusing text and structure knowledge on sparse knowledge graph completion

• Main Authors: HE, Tao, LIU, Ming, CAO, Yixin, QU, Meng, ZHENG, Zihao, QIN, Bing
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Centralized optimization; data-driven optimization; distributed optimization; evolutionary computation; privacy protection; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/8664
• Institution: Singapore Management University

The task of Knowledge Graph Completion (KGC) is to infer missing links for Knowledge Graphs (KGs) by analyzing graph structures. However, with increasing sparsity in KGs, this task becomes increasingly challenging. In this paper, we propose VEM2L, a joint learning framework that incorporates structure and relevant text information to supplement insufcient features for sparse KGs. We begin by training two pre-existing KGC models: one based on structure and the other based on text. Our ultimate goal is to fuse knowledge acquired by these models. To achieve this, we divide knowledge within the models into two non-overlapping parts: expressive power and generalization ability. We then propose two diferent joint learning methods that co-distill these two kinds of knowledge respectively. For expressive power, we allow each model to learn from and exchange knowledge mutually on training examples. For the generalization ability, we propose a novel co-distillation strategy using the Variational EM algorithm on unobserved queries. Our proposed joint learning framework is supported by both detailed theoretical evidence and qualitative experiments, demonstrating its efectiveness.