Prompt to be consistent is better than self-consistent? Few-shot and zero-shot fact verification with pre-trained language models

• Main Authors: ZENG, Fengzhu, GAO, Wei
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: few-shot fact verification; zero-shot fact verification; ProToCo; Prompt pre-trained language models; factuality assessment; claim-evidence pair; consistency mechanism; variants; predictions; parameter-efficient fine-tuning (PEFT); accurate predictions; public verification datasets; zero-shot learner; in-context learning (ICL); OPT-30B; Self-Consistency-enabled OPT-6.7B model; Computer Sciences
• Online Access: https://ink.library.smu.edu.sg/sis_research/8452; https://ink.library.smu.edu.sg/context/sis_research/article/9455/viewcontent/2023.findings_acl.278.pdf
• Institution: Singapore Management University

Few-shot or zero-shot fact verification only relies on a few or no labeled training examples. In this paper, we propose a novel method called ProToCo, to Prompt pre-trained language models (PLMs) To be Consistent, for improving the factuality assessment capability of PLMs in the few-shot and zero-shot settings. Given a claim-evidence pair, ProToCo generates multiple variants of the claim with different relations and frames a simple consistency mechanism as constraints for making compatible predictions across these variants. We update PLMs by using parameter-efficient fine-tuning (PEFT), leading to more accurate predictions in few-shot and zero-shot fact verification tasks. Our experiments on three public verification datasets show that ProToCo significantly outperforms state-of-the-art few-shot fact verification baselines. With a small number of unlabeled instances, ProToCo also outperforms the strong zero-shot learner T0 on zero-shot verification. Compared to large PLMs using in-context learning (ICL) method, ProToCo outperforms OPT-30B and the Self-Consistency-enabled OPT-6.7B model in both few- and zero-shot settings.