NumGPT: Improving numeracy ability of generative pre-trained models

• Main Authors: JIN, Zhihua, JIANG, Xin, WANG, Xiangbo, LIU, Qun, WANG, Yong, REN, Xiaozhe, QU, Huamin
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Artificial Intelligence and Robotics
• Online Access: https://ink.library.smu.edu.sg/sis_research/8599; https://ink.library.smu.edu.sg/context/sis_research/article/9602/viewcontent/2109.03137.pdf
• Institution: Singapore Management University

Existing generative pre-trained language models (e.g., GPT) focus on modeling the language structure and semantics of general texts. However, those models do not consider the numerical properties of numbers and cannot perform robustly on numerical reasoning tasks (e.g., math word problems and measur...