Probing effects of contextual bias on number magnitude estimation

• Main Authors: DU, Xuehao, JI, Ping, QIN, Wei, WANG, Lei, LAN, Yunshi
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Natural language processing; Question answering; Out of distribution; Contextual bias; Number magnitude estimation; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/9428; https://ink.library.smu.edu.sg/context/sis_research/article/10428/viewcontent/journal_tiis_18_9_131970802_pvoa.pdf
• Institution: Singapore Management University

The semantic understanding of numbers requires association with context. However, powerful neural networks overfit spurious correlations between context and numbers in training corpus can lead to the occurrence of contextual bias, which may affect the network's accurate estimation of number magnitude when making inferences in real-world data. To investigate the resilience of current methodologies against contextual bias, we introduce a novel out-of- distribution (OOD) numerical question-answering (QA) dataset that features specific correlations between context and numbers in the training data, which are not present in the OOD test data. We evaluate the robustness of different numerical encoding and decoding methods when confronted with contextual bias on this dataset. Our findings indicate that encoding methods incorporating more detailed digit information exhibit greater resilience against contextual bias. Inspired by this finding, we propose a digit-aware position embedding strategy, and the experimental results demonstrate that this strategy is highly effective in improving the robustness of neural networks against contextual bias.