Large-scale correlation analysis of automated metrics for topic models

• Main Authors: LIM, Jia Peng, LAUW, Hady Wirawan
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Automated metric; Coherence metric; Correlation analysis; Human decision-making; Human judgments; Large corpora; Large-scale correlations; Metric evaluation; Topic Modeling; User study; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/8346; https://ink.library.smu.edu.sg/context/sis_research/article/9349/viewcontent/acl23.pdf
• Institution: Singapore Management University

Automated coherence metrics constitute an important and popular way to evaluate topic models. Previous works present a mixed picture of their presumed correlation with human judgement. In this paper, we conduct a large-scale correlation analysis of coherence metrics. We propose a novel sampling appr...