A Prototype of a Conversational Virtual University Support Agent Powered by a Large Language Model That Addresses Inquiries About Policies in the Student Handbook

• Main Authors: Ilagan, Joseph Benjamin R, Ilagan, Jose Ramon
• Format: text
• Published: Archīum Ateneo 2024
• Subjects: Chatbot; Conversational agents; Conversational User Experience; Large language model; Natural language processing; Artificial Intelligence and Robotics; Computer Sciences; Physical Sciences and Mathematics
• Online Access: https://archium.ateneo.edu/qmit-faculty-pubs/25; https://archium.ateneo.edu/context/qmit-faculty-pubs/article/1024/viewcontent/1_s2.0_S1877050924015217_main.pdf
• Institution: Ateneo De Manila University

Universities gain a competitive advantage by deliberately improving overall service, student, faculty, and staff experience, leading to attractiveness, retention, and improved outcomes. Quality services are achieved partly by addressing employee satisfaction, specifically in the work environment. This paper presents a prototype study of a virtual university support agent, a system grounded in a Large Language Model (LLM) engineered to address inquiries from university students, faculty and staff related to the student handbook. The study investigates the integration of generative artificial intelligence and natural conversation properties inherent in LLMs to overcome customer service shortcomings identified in previous chatbot applications. The LLMs' susceptibility to 'hallucination' is mitigated through a combined approach of few-shot learning and chain of thought libraries in the training phase. The information core of this system comprises student handbook PDF files, from which an algorithm extracts and structures data to be utilized by the LLM. As a result, the university support agent facilitates a viable Q&A interface for students, faculty, and administrators to inquire about university guidelines and policies.