Semantic matching in machine reading comprehension: an empirical study
Machine reading comprehension (MRC) is a challenging task in the field of artificial intelligence. Most existing MRC works contain a semantic matching module, either explicitly or intrinsically, to determine whether a piece of context answers a question. However, there is scant work which systematic...
Saved in:
Main Authors: | , , , |
---|---|
Other Authors: | |
Format: | Article |
Language: | English |
Published: |
2023
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/164704 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-164704 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1647042023-02-10T06:13:15Z Semantic matching in machine reading comprehension: an empirical study Liu, Qian Mao, Rui Geng, Xiubo Cambria, Erik School of Computer Science and Engineering Engineering::Computer science and engineering Natural Language Processing Machine Reading Comprehension Machine reading comprehension (MRC) is a challenging task in the field of artificial intelligence. Most existing MRC works contain a semantic matching module, either explicitly or intrinsically, to determine whether a piece of context answers a question. However, there is scant work which systematically evaluates different paradigms using semantic matching in MRC. In this paper, we conduct a systematic empirical study on semantic matching. We formulate a two-stage framework which consists of a semantic matching model and a reading model, based on pre-trained language models. We compare and analyze the effectiveness and efficiency of using semantic matching modules with different setups on four types of MRC datasets. We verify that using semantic matching before a reading model improves both the effectiveness and efficiency of MRC. Compared with answering questions by extracting information from concise context, we observe that semantic matching yields more improvements for answering questions with noisy and adversarial context. Matching coarse-grained context to questions, e.g., paragraphs, is more effective than matching fine-grained context, e.g., sentences and spans. We also find that semantic matching is helpful for answering who/where/when/what/how/which questions, whereas it decreases the MRC performance on why questions. This may imply that semantic matching helps to answer a question whose necessary information can be retrieved from a single sentence. The above observations demonstrate the advantages and disadvantages of using semantic matching in different scenarios. Agency for Science, Technology and Research (A*STAR) This research is supported by the Agency for Science, Technology and Research (A*STAR), Singapore under its AME Programmatic Funding Scheme (Project #A18A2b0046). 2023-02-10T06:13:15Z 2023-02-10T06:13:15Z 2023 Journal Article Liu, Q., Mao, R., Geng, X. & Cambria, E. (2023). Semantic matching in machine reading comprehension: an empirical study. Information Processing and Management, 60(2), 103145-. https://dx.doi.org/10.1016/j.ipm.2022.103145 0306-4573 https://hdl.handle.net/10356/164704 10.1016/j.ipm.2022.103145 2-s2.0-85142338736 2 60 103145 en A18A2b0046 Information Processing and Management © 2022 Elsevier Ltd. All rights reserved. |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering::Computer science and engineering Natural Language Processing Machine Reading Comprehension |
spellingShingle |
Engineering::Computer science and engineering Natural Language Processing Machine Reading Comprehension Liu, Qian Mao, Rui Geng, Xiubo Cambria, Erik Semantic matching in machine reading comprehension: an empirical study |
description |
Machine reading comprehension (MRC) is a challenging task in the field of artificial intelligence. Most existing MRC works contain a semantic matching module, either explicitly or intrinsically, to determine whether a piece of context answers a question. However, there is scant work which systematically evaluates different paradigms using semantic matching in MRC. In this paper, we conduct a systematic empirical study on semantic matching. We formulate a two-stage framework which consists of a semantic matching model and a reading model, based on pre-trained language models. We compare and analyze the effectiveness and efficiency of using semantic matching modules with different setups on four types of MRC datasets. We verify that using semantic matching before a reading model improves both the effectiveness and efficiency of MRC. Compared with answering questions by extracting information from concise context, we observe that semantic matching yields more improvements for answering questions with noisy and adversarial context. Matching coarse-grained context to questions, e.g., paragraphs, is more effective than matching fine-grained context, e.g., sentences and spans. We also find that semantic matching is helpful for answering who/where/when/what/how/which questions, whereas it decreases the MRC performance on why questions. This may imply that semantic matching helps to answer a question whose necessary information can be retrieved from a single sentence. The above observations demonstrate the advantages and disadvantages of using semantic matching in different scenarios. |
author2 |
School of Computer Science and Engineering |
author_facet |
School of Computer Science and Engineering Liu, Qian Mao, Rui Geng, Xiubo Cambria, Erik |
format |
Article |
author |
Liu, Qian Mao, Rui Geng, Xiubo Cambria, Erik |
author_sort |
Liu, Qian |
title |
Semantic matching in machine reading comprehension: an empirical study |
title_short |
Semantic matching in machine reading comprehension: an empirical study |
title_full |
Semantic matching in machine reading comprehension: an empirical study |
title_fullStr |
Semantic matching in machine reading comprehension: an empirical study |
title_full_unstemmed |
Semantic matching in machine reading comprehension: an empirical study |
title_sort |
semantic matching in machine reading comprehension: an empirical study |
publishDate |
2023 |
url |
https://hdl.handle.net/10356/164704 |
_version_ |
1759058815161991168 |