Disentangling transformer language models as superposed topic models

• Main Authors: LIM, Jia Peng, LAUW, Hady Wirawan
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Databases and Information Systems; Programming Languages and Compilers
• Online Access: https://ink.library.smu.edu.sg/sis_research/8470; https://ink.library.smu.edu.sg/context/sis_research/article/9473/viewcontent/2023.emnlp_main.534__1_.pdf
• Institution: Singapore Management University

Topic Modelling is an established research area where the quality of a given topic is measured using coherence metrics. Often, we infer topics from Neural Topic Models (NTM) by interpreting their decoder weights, consisting of top-activated words projected from individual neurons. Transformer-based...