Automating dataset updates towards reliable and timely evaluation of Large Language Models

• Main Authors: YING, Jiahao, CAO, Yixin, BAI, Yushi, SUN, Qianru, WANG, Bo, TANG, Wei, DING, Zhaojun, YANG, Yizhe, HUANG, Xuanjing, YAN, Shuicheng
• 格式: text
• 語言: English
• 出版: Institutional Knowledge at Singapore Management University 2024
• 主題: Large language models; LLM; Dataset update; Benchmark update; Automation; Artificial Intelligence and Robotics
• 在線閱讀: https://ink.library.smu.edu.sg/sis_research/9439
• 機構: Singapore Management University

Large language models (LLMs) have achieved impressive performance across various natural language benchmarks, prompting a continual need to curate more difficult datasets for larger LLMs, which is costly and time-consuming. In this paper, we propose to automate dataset updating and provide systemati...