Towards explainable and semantically coherent claim extraction for an automated fact-checker
Misinformation and fake news spread everywhere through online social media platforms. Although Automatic Fact-Checkers and LLMs like ChatGPT have become popular and seem to be a promising solution to detect fake news, these models still have some limitations regarding their reliance on pre-existing...
Saved in:
Main Author: | |
---|---|
Other Authors: | |
Format: | Final Year Project |
Language: | English |
Published: |
Nanyang Technological University
2024
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/176481 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-176481 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1764812024-05-17T15:45:38Z Towards explainable and semantically coherent claim extraction for an automated fact-checker Yoswara, Jocelyn Valencia Erry Gunawan School of Electrical and Electronic Engineering Institute for Infocomm Research, A*STAR EGUNAWAN@ntu.edu.sg Computer and Information Science Engineering Natural language processing Large language models Claim extraction Automatic fact checker Machine learning Misinformation and fake news spread everywhere through online social media platforms. Although Automatic Fact-Checkers and LLMs like ChatGPT have become popular and seem to be a promising solution to detect fake news, these models still have some limitations regarding their reliance on pre-existing knowledge, and concerns are raised about whether they can differentiate between truth and falsehood. As these concerns arise, the claim extraction process of obtaining claims made from various resources becomes one of the most crucial steps in an automatic fact-checker. Therefore, this project aims to enhance the claim extraction process of an automatic fact-checker, improve the accuracy of fact-checking, and mitigate the limitations associated with LLMs' reliance on outdated information. Firstly, a baseline model was created using SBert with an evidence retrieval system. Secondly, an evidence retrieval system was implemented using the Google Search API to gather evidence related to claims from online sources. Lastly, the GPT-4 model was utilized to verify claims based on the available evidence. The GPT-4 model outperforms the baseline model, achieving a 94% accuracy in claim verification. However, the baseline model provides more comprehensive insights into the coverage of the entire dataset, and it is also found that the evidence retrieval process significantly affects the model's accuracy and coverage of claim verification tasks. Hence, this project represents an enhancement in the claim extraction process while also identifying limitations and suggesting potential areas for further improvements to improve the effectiveness of claim extraction for an automatic fact-checker. Bachelor's degree 2024-05-17T02:20:17Z 2024-05-17T02:20:17Z 2024 Final Year Project (FYP) Yoswara, J. V. (2024). Towards explainable and semantically coherent claim extraction for an automated fact-checker. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/176481 https://hdl.handle.net/10356/176481 en B3054 -231 application/pdf Nanyang Technological University |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Computer and Information Science Engineering Natural language processing Large language models Claim extraction Automatic fact checker Machine learning |
spellingShingle |
Computer and Information Science Engineering Natural language processing Large language models Claim extraction Automatic fact checker Machine learning Yoswara, Jocelyn Valencia Towards explainable and semantically coherent claim extraction for an automated fact-checker |
description |
Misinformation and fake news spread everywhere through online social media platforms. Although Automatic Fact-Checkers and LLMs like ChatGPT have become popular and seem to be a promising solution to detect fake news, these models still have some limitations regarding their reliance on pre-existing knowledge, and concerns are raised about whether they can differentiate between truth and falsehood. As these concerns arise, the claim extraction process of obtaining claims made from various resources becomes one of the most crucial steps in an automatic fact-checker. Therefore, this project aims to enhance the claim extraction process of an automatic fact-checker, improve the accuracy of fact-checking, and mitigate the limitations associated with LLMs' reliance on outdated information.
Firstly, a baseline model was created using SBert with an evidence retrieval system. Secondly, an evidence retrieval system was implemented using the Google Search API to gather evidence related to claims from online sources. Lastly, the GPT-4 model was utilized to verify claims based on the available evidence. The GPT-4 model outperforms the baseline model, achieving a 94% accuracy in claim verification. However, the baseline model provides more comprehensive insights into the coverage of the entire dataset, and it is also found that the evidence retrieval process significantly affects the model's accuracy and coverage of claim verification tasks. Hence, this project represents an enhancement in the claim extraction process while also identifying limitations and suggesting potential areas for further improvements to improve the effectiveness of claim extraction for an automatic fact-checker. |
author2 |
Erry Gunawan |
author_facet |
Erry Gunawan Yoswara, Jocelyn Valencia |
format |
Final Year Project |
author |
Yoswara, Jocelyn Valencia |
author_sort |
Yoswara, Jocelyn Valencia |
title |
Towards explainable and semantically coherent claim extraction for an automated fact-checker |
title_short |
Towards explainable and semantically coherent claim extraction for an automated fact-checker |
title_full |
Towards explainable and semantically coherent claim extraction for an automated fact-checker |
title_fullStr |
Towards explainable and semantically coherent claim extraction for an automated fact-checker |
title_full_unstemmed |
Towards explainable and semantically coherent claim extraction for an automated fact-checker |
title_sort |
towards explainable and semantically coherent claim extraction for an automated fact-checker |
publisher |
Nanyang Technological University |
publishDate |
2024 |
url |
https://hdl.handle.net/10356/176481 |
_version_ |
1814047191172382720 |