Knowledge enhanced stance detection

Stance detection is a task that aims to identify the stance expressed in a document towards a specific target. Recent advancements, like Chain-of-Thought (CoT) prompting, have enhanced the reasoning capabilities of Large Language Models (LLMs) by incorporating intermediate rationales. However, the e...

Full description

Saved in:
Bibliographic Details
Main Author: Hu, Kairui
Other Authors: Guan Cuntai
Format: Final Year Project
Language:English
Published: Nanyang Technological University 2024
Subjects:
Online Access:https://hdl.handle.net/10356/175100
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-175100
record_format dspace
spelling sg-ntu-dr.10356-1751002024-04-19T15:42:25Z Knowledge enhanced stance detection Hu, Kairui Guan Cuntai School of Computer Science and Engineering CTGuan@ntu.edu.sg Computer and Information Science Stance detection Large language models Knowledge enhancement Natural language processing Stance detection is a task that aims to identify the stance expressed in a document towards a specific target. Recent advancements, like Chain-of-Thought (CoT) prompting, have enhanced the reasoning capabilities of Large Language Models (LLMs) by incorporating intermediate rationales. However, the efficacy of CoT is constrained by the model's limited internal knowledge, which often leads to an inaccurate understanding and subsequently undermines the stance prediction. This limitation might further result in hallucinations, where LLMs produce unfaithful responses and erroneous reasoning, compromising the output's reliability and precision. Moreover, CoT struggles to perform effectively on smaller language models with inadequate knowledge and reasoning capabilities, raising concerns on efficiency. To address these issues, we introduce the Ladder-of-Thought (LoT), a novel method using knowledge as steps to elevate stance detection. LoT implements a triple-phase Progressive Optimization Framework: 1) External Knowledge Injection, where the model's knowledge base is expanded; 2) Intermediate Knowledge Generation, which produces more reliable intermediate knowledge to enhance prediction; and 3) Downstream Fine-tuning & Prediction, improving the model's prediction accuracy. This sequential approach symbolizes ascending a ladder, with each phase representing a progressive step towards achieving optimal reasoning and prediction performance. Our empirical results demonstrate that LoT achieves state-of-the-art results in zero-shot/few-shot and in-target stance detection, marking a 16% improvement over ChatGPT and a 10% enhancement compared to ChatGPT with CoT on stance detection task. Bachelor's degree 2024-04-19T05:43:50Z 2024-04-19T05:43:50Z 2024 Final Year Project (FYP) Hu, K. (2024). Knowledge enhanced stance detection. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/175100 https://hdl.handle.net/10356/175100 en SCSE23-0155 application/pdf Nanyang Technological University
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Computer and Information Science
Stance detection
Large language models
Knowledge enhancement
Natural language processing
spellingShingle Computer and Information Science
Stance detection
Large language models
Knowledge enhancement
Natural language processing
Hu, Kairui
Knowledge enhanced stance detection
description Stance detection is a task that aims to identify the stance expressed in a document towards a specific target. Recent advancements, like Chain-of-Thought (CoT) prompting, have enhanced the reasoning capabilities of Large Language Models (LLMs) by incorporating intermediate rationales. However, the efficacy of CoT is constrained by the model's limited internal knowledge, which often leads to an inaccurate understanding and subsequently undermines the stance prediction. This limitation might further result in hallucinations, where LLMs produce unfaithful responses and erroneous reasoning, compromising the output's reliability and precision. Moreover, CoT struggles to perform effectively on smaller language models with inadequate knowledge and reasoning capabilities, raising concerns on efficiency. To address these issues, we introduce the Ladder-of-Thought (LoT), a novel method using knowledge as steps to elevate stance detection. LoT implements a triple-phase Progressive Optimization Framework: 1) External Knowledge Injection, where the model's knowledge base is expanded; 2) Intermediate Knowledge Generation, which produces more reliable intermediate knowledge to enhance prediction; and 3) Downstream Fine-tuning & Prediction, improving the model's prediction accuracy. This sequential approach symbolizes ascending a ladder, with each phase representing a progressive step towards achieving optimal reasoning and prediction performance. Our empirical results demonstrate that LoT achieves state-of-the-art results in zero-shot/few-shot and in-target stance detection, marking a 16% improvement over ChatGPT and a 10% enhancement compared to ChatGPT with CoT on stance detection task.
author2 Guan Cuntai
author_facet Guan Cuntai
Hu, Kairui
format Final Year Project
author Hu, Kairui
author_sort Hu, Kairui
title Knowledge enhanced stance detection
title_short Knowledge enhanced stance detection
title_full Knowledge enhanced stance detection
title_fullStr Knowledge enhanced stance detection
title_full_unstemmed Knowledge enhanced stance detection
title_sort knowledge enhanced stance detection
publisher Nanyang Technological University
publishDate 2024
url https://hdl.handle.net/10356/175100
_version_ 1800916299968151552