Benchmarking foundation models with language-model-as-an-examiner

• Main Authors: BAI, Yushi, YING, Jiahao, CAO, Yixin, LV, Xin, HE, Yuze, WANG, Xiaozhi, YU, Jifan, ZENG, Kaisheng, XIAO, Yijia, LYU, Haozhe, ZHANG, Jiayin, LI, Juanzi, HOU, Lei
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Databases and Information Systems; Programming Languages and Compilers
• Online Access: https://ink.library.smu.edu.sg/sis_research/8392; https://ink.library.smu.edu.sg/context/sis_research/article/9395/viewcontent/2306.04181.pdf
• Institution: Singapore Management University

Numerous benchmarks have been established to assess the performance of foundation models on open-ended question answering, which serves as a comprehensive test of a model’s ability to understand and generate language in a manner similar to humans. Most of these works focus on proposing new datasets,...