Joint representation learning of cross-lingual words and entities via attentive distant supervision

• Main Authors: CAO, Yixin, HOU, Lei, LI, Juanzi, LIU, Zhiyuan, LI, Chengjiang, CHEN, Xu, DONG, Tiansi
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2018
• Subjects: Databases and Information Systems; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/7465; https://ink.library.smu.edu.sg/context/sis_research/article/8468/viewcontent/D18_1021.pdf
• Institution: Singapore Management University

Joint representation learning of words and entities benefits many NLP tasks, but has not been well explored in cross-lingual settings. In this paper, we propose a novel method for joint representation learning of cross-lingual words and entities. It captures mutually complementary knowledge, and enables cross-lingual inferences among knowledge bases and texts. Our method does not require parallel corpora, and automatically generates comparable data via distant supervision using multi-lingual knowledge bases. We utilize two types of regularizers to align cross-lingual words and entities, and design knowledge attention and crosslingual attention to further reduce noises. We conducted a series of experiments on three tasks: word translation, entity relatedness, and cross-lingual entity linking. The results, both qualitatively and quantitatively, demonstrate the significance of our method.