Active learning for ontological event extraction incorporating named entity recognition and unknown word handling

Background: Biomedical text mining may target various kinds of valuable information embedded in the literature, but a critical obstacle to the extension of the mining targets is the cost of manual construction of labeled data, which are required for state-of-the-art supervised learning systems. Acti...

Full description

Saved in:
Bibliographic Details
Main Authors: Han, Xu, Kim, Jung-jae, Kwoh, Chee Keong
Other Authors: School of Computer Science and Engineering
Format: Article
Language:English
Published: 2016
Subjects:
Online Access:https://hdl.handle.net/10356/81505
http://hdl.handle.net/10220/40808
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-81505
record_format dspace
spelling sg-ntu-dr.10356-815052022-02-16T16:26:34Z Active learning for ontological event extraction incorporating named entity recognition and unknown word handling Han, Xu Kim, Jung-jae Kwoh, Chee Keong School of Computer Science and Engineering Active learning Biomedical natural language processing Information extraction Background: Biomedical text mining may target various kinds of valuable information embedded in the literature, but a critical obstacle to the extension of the mining targets is the cost of manual construction of labeled data, which are required for state-of-the-art supervised learning systems. Active learning is to choose the most informative documents for the supervised learning in order to reduce the amount of required manual annotations. Previous works of active learning, however, focused on the tasks of entity recognition and protein-protein interactions, but not on event extraction tasks for multiple event types. They also did not consider the evidence of event participants, which might be a clue for the presence of events in unlabeled documents. Moreover, the confidence scores of events produced by event extraction systems are not reliable for ranking documents in terms of informativity for supervised learning. We here propose a novel committee-based active learning method that supports multi-event extraction tasks and employs a new statistical method for informativity estimation instead of using the confidence scores from event extraction systems. Methods: Our method is based on a committee of two systems as follows: We first employ an event extraction system to filter potential false negatives among unlabeled documents, from which the system does not extract any event. We then develop a statistical method to rank the potential false negatives of unlabeled documents 1) by using a language model that measures the probabilities of the expression of multiple events in documents and 2) by using a named entity recognition system that locates the named entities that can be event arguments (e.g. proteins). The proposed method further deals with unknown words in test data by using word similarity measures. We also apply our active learning method for the task of named entity recognition. Results and conclusion: We evaluate the proposed method against the BioNLP Shared Tasks datasets, and show that our method can achieve better performance than such previous methods as entropy and Gibbs error based methods and a conventional committee-based method. We also show that the incorporation of named entity recognition into the active learning for event extraction and the unknown word handling further improve the active learning method. In addition, the adaptation of the active learning method into named entity recognition tasks also improves the document selection for manual annotation of named entities. MOE (Min. of Education, S’pore) Published version 2016-06-27T08:56:53Z 2019-12-06T14:32:30Z 2016-06-27T08:56:53Z 2019-12-06T14:32:30Z 2016 Journal Article Han, X., Kim, J. j., & Kwoh, C. K. (2016). Active learning for ontological event extraction incorporating named entity recognition and unknown word handling. Journal of Biomedical Semantics, 7(1), 22-. 2041-1480 https://hdl.handle.net/10356/81505 http://hdl.handle.net/10220/40808 10.1186/s13326-016-0059-z 27127603 en Journal of Biomedical Semantics © 2016 Han et al. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. 18 p. application/pdf
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Active learning
Biomedical natural language processing
Information extraction
spellingShingle Active learning
Biomedical natural language processing
Information extraction
Han, Xu
Kim, Jung-jae
Kwoh, Chee Keong
Active learning for ontological event extraction incorporating named entity recognition and unknown word handling
description Background: Biomedical text mining may target various kinds of valuable information embedded in the literature, but a critical obstacle to the extension of the mining targets is the cost of manual construction of labeled data, which are required for state-of-the-art supervised learning systems. Active learning is to choose the most informative documents for the supervised learning in order to reduce the amount of required manual annotations. Previous works of active learning, however, focused on the tasks of entity recognition and protein-protein interactions, but not on event extraction tasks for multiple event types. They also did not consider the evidence of event participants, which might be a clue for the presence of events in unlabeled documents. Moreover, the confidence scores of events produced by event extraction systems are not reliable for ranking documents in terms of informativity for supervised learning. We here propose a novel committee-based active learning method that supports multi-event extraction tasks and employs a new statistical method for informativity estimation instead of using the confidence scores from event extraction systems. Methods: Our method is based on a committee of two systems as follows: We first employ an event extraction system to filter potential false negatives among unlabeled documents, from which the system does not extract any event. We then develop a statistical method to rank the potential false negatives of unlabeled documents 1) by using a language model that measures the probabilities of the expression of multiple events in documents and 2) by using a named entity recognition system that locates the named entities that can be event arguments (e.g. proteins). The proposed method further deals with unknown words in test data by using word similarity measures. We also apply our active learning method for the task of named entity recognition. Results and conclusion: We evaluate the proposed method against the BioNLP Shared Tasks datasets, and show that our method can achieve better performance than such previous methods as entropy and Gibbs error based methods and a conventional committee-based method. We also show that the incorporation of named entity recognition into the active learning for event extraction and the unknown word handling further improve the active learning method. In addition, the adaptation of the active learning method into named entity recognition tasks also improves the document selection for manual annotation of named entities.
author2 School of Computer Science and Engineering
author_facet School of Computer Science and Engineering
Han, Xu
Kim, Jung-jae
Kwoh, Chee Keong
format Article
author Han, Xu
Kim, Jung-jae
Kwoh, Chee Keong
author_sort Han, Xu
title Active learning for ontological event extraction incorporating named entity recognition and unknown word handling
title_short Active learning for ontological event extraction incorporating named entity recognition and unknown word handling
title_full Active learning for ontological event extraction incorporating named entity recognition and unknown word handling
title_fullStr Active learning for ontological event extraction incorporating named entity recognition and unknown word handling
title_full_unstemmed Active learning for ontological event extraction incorporating named entity recognition and unknown word handling
title_sort active learning for ontological event extraction incorporating named entity recognition and unknown word handling
publishDate 2016
url https://hdl.handle.net/10356/81505
http://hdl.handle.net/10220/40808
_version_ 1725985770209017856