CLAMBER: A benchmark of identifying and clarifying ambiguous information needs in large language models

• Main Authors: ZHANG, Tong, QIN, Peixin, DENG, Yang, HUANG, Chen, LEI, Wenqiang, LIU, Junhong, JIN, Dingnan, LIANG, Hongru, CHUA, Tat-Seng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Databases and Information Systems; Programming Languages and Compilers
• Online Access: https://ink.library.smu.edu.sg/sis_research/9238; https://ink.library.smu.edu.sg/context/sis_research/article/10238/viewcontent/2024.acl_long.578.pdf
• Institution: Singapore Management University

Large language models (LLMs) are increasingly used to meet user information needs, but their effectiveness in dealing with user queries that contain various types of ambiguity remains unknown, ultimately risking user trust and satisfaction. To this end, we introduce CLAMBER, a benchmark for evaluati...