Fake review detection by fusing parameter efficient adapters in pre-trained language model

• Main Author: Ho, See Cheng
• Other Authors: Lihui Chen
• Format: Thesis-Master by Research
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/173129
• Institution: Nanyang Technological University

Peer-to-peer reviews are important to businesses. Review ratings affect the reputations of businesses, which helps the growth of the business. However, fake reviews are increasingly plaguing the internet, leading to bad quality purchases or even scams. This is especially common on services and marketplace platforms such as Yelp, TripAdvisor and Amazon, whereby customers rely heavily on reviews on these platforms before paying for items or services from businesses on the platforms. Therefore, developing a system to detect fake reviews written by bad actors, is of utmost importance to protect the integrity of both platforms and businesses. Currently, there are many deep learning models utilizing large pre-trained language models to address the problem by analyzing text data. However, the identifiable pattern of fake reviews tends to change rapidly, resulting in the necessity of updating these models frequently. Large pre-trained language models usually have a huge number of parameters, which poses a challenge when it comes to retraining them periodically due to the large computes needed and the problem of catastrophic inference. To address this problem, this thesis utilizes adapter, a small set of parameters that is inserted into a transformer language model. A set of adapters will be fine-tuned to solve the fake review tasks, taking advantage of them being compact, modular, and composable modules. This allows the pre-trained models to retain their knowledge and reduces the memory storage required to store knowledge of various downstream tasks. In addition, multiple adapters can be fused together using the AdapterFusion methodology, opening additional solutions to introduce useful external knowledge into the model. In our experiments, we observe that using adapters achieve comparable performance to a fully fine-tuned language model for fake review detection. Additionally, by fusing adapters, with the introduction of external knowledge such as contextualized emotion and sentiment knowledge, we improve the model further, while reducing storage utilization and improving parameter efficiency. The results highlight the challenge of fake review detection and the need to explore solutions for efficiency instead of focusing deeper models.