DocEE: A large-scale and fine-grained benchmark for document-level event extraction

• Main Authors: TONG, Meihan, XU, Bin, WANG, Shuai, HAN, Meihuan, CAO, Yixin, ZHU, Jiangqi, CHEN, Siyu, HOU, Lei, LI, Juanzi
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Databases and Information Systems; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/7471; https://ink.library.smu.edu.sg/context/sis_research/article/8474/viewcontent/2022.naacl_main.291.pdf
• Institution: Singapore Management University

Event extraction aims to identify an event and then extract the arguments participating in the event. Despite the great success in sentencelevel event extraction, events are more naturally presented in the form of documents, with event arguments scattered in multiple sentences. However, a major barrier to promote documentlevel event extraction has been the lack of large-scale and practical training and evaluation datasets. In this paper, we present DocEE, a new document-level event extraction dataset including 27,000+ events, 180,000+ arguments. We highlight three features: largescale manual annotations, fine-grained argument types and application-oriented settings. Experiments show that there is still a big gap between state-of-the-art models and human beings (41% Vs 85% in F1 score), indicating that DocEE is an open issue. DocEE is now available at https://github.com/ tongmeihan1995/DocEE.git.