ChatGPT and its robustness, fairness, trustworthiness and impact

• Main Author: Muhammad Akmal Bin Rahmat
• Other Authors: Alex Chichung Kot
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering; ChatGPT
• Online Access: https://hdl.handle.net/10356/181710
• Institution: Nanyang Technological University

The swift progress of AI has brought about a new age of LLMs, with models such as ChatGPT-4 leading the way in these advancements. With the integration of different types of inputs like text, images, and other data, managing robustness, fairness, trustworthiness and hallucinations in models becomes more difficult when the generated information is not accurately connected to the input provided. This report delves into examining the resilience, equity, and reliability of ChatGPT-4. The main goal of this study is to analyze the occurrence of inconsistent, unfair and untrustworthy answers and hallucinations in ChatGPT-4 when handling multimodal inputs and to examine the effectiveness of Factually Augmented Reinforcement Learning from Human Feedback (RLHF) as a possible remedy. The research includes an in-depth examination of the pertinent literature on LLMs, RLHF, and multimodal hallucinations, paired with an empirical evaluation utilizing a dataset of images and accompanying textual replies produced by ChatGPT-4. Important discoveries show that although ChatGPT-4 shows prowess in some respects like recognizing its limitations and giving precise quantitative information, it struggles with complex comparative analyses and providing thorough contextual details. The use of Factually Augmented RLHF offers potential in tackling these problems by including more factual information and clearer reward signals, leading to a decrease in hallucinations and improvement in overall model performance. This report enhances our knowledge of AI robustness and fairness by exploring the unique difficulties of multimodal hallucinations and assessing possible ways to address them. The results highlight the importance of continued research and improvement in AI systems to guarantee their dependability and credibility in practical settings.