Leveraging large language models for effective user interaction via conversations

• Main Author: Zhang, Mengao
• Other Authors: Li Boyang
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Artificial intelligence; Natural language processing; Explainable AI; Large language model; Deep learning
• Online Access: https://hdl.handle.net/10356/175241
• Institution: Nanyang Technological University

The field of eXplainable AI (XAI) aims to clarify the decision-making processes of black-box AI models for human comprehension. Current XAI approaches mainly rely on static explanations, often failing to accommodate the diverse backgrounds and varying levels of user understanding. Recent studies have shown the effectiveness of incorporating conversational interactions along with static explanations in enhancing user understanding and trust in AI models. However, the time and domain knowledge required to respond to user queries makes this approach costly. This report presents an approach that leverages large language models (LLMs) to provide dynamic, user-centric explanations through free-form conversations. The methodology involves prompt engineering, data generation using LLMs, and fine-tuning LLMs on the generated data. Experiments were conducted on image classification tasks, evaluating the performance of the fine-tuned models using both objective quantitative metrics and subjective human evaluations. The results demonstrate that fine-tuning LLMs with generated data can improve their performance, particularly when incorporating real examples and diverse data. Furthermore, we propose an iterative process of data generation and fine-tuning that leads to even further performance improvements. This work contributes to the development of user-centric, conversational XAI systems, enabling more effective communication and understanding between AI models and users with varying backgrounds and needs.