Impact of difficult negatives on Twitter crisis detection

• Main Authors: ZHANG, Yuhao, LO, Siaw Ling, WIN MYINT, Phyo Yi
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Twitter; Crisis Detection; Difficult Negative Data; Negative Mining; Databases and Information Systems; Social Media
• Online Access: https://ink.library.smu.edu.sg/sis_research/8007; https://ink.library.smu.edu.sg/context/sis_research/article/9010/viewcontent/LoSiawLing_2023_Impact_of_Difficult_Negatives_on_Twitter_Crisis_Detection.pdf
• Institution: Singapore Management University

Twitter has become an alternative information source during a crisis. However, the short, noisy nature of tweets hinders information extraction. While models trained with standard Twitter crisis datasets accomplished decent performance, it remained a challenge to generalize to unseen crisis events. Thus, we proposed adding “difficult” negative examples during training to improve model generalization for Twitter crisis detection. Although adding random noise is a common practice, the impact of difficult negatives, i.e., negative data semantically similar to true examples, was never examined in NLP. Most of existing research focuses on the classification task, without considering the primary information need of crisis responders. In our study, we implemented multiple sequence tagging models and studied quantitatively and qualitatively the impact of difficult negatives on sequence tagging. We evaluated models on unseen events and showed that difficult negative forced models to generalize better, leading to more accurate information extraction in a real-world application.