Towards human-level artificial intelligence agents

• Main Author: Leung, Jonathan Cyril
• Other Authors: Miao Chun Yan
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Goal modelling; Reinforcement learning; Agent development; Machine learning
• Online Access: https://hdl.handle.net/10356/174532
• Institution: Nanyang Technological University

Deep learning has provided a method to train large neural networks to learn a representation of data that best solves a given task without the need for manual feature engineering. The combination of Reinforcement Learning (RL) and deep learning, often referred to as Deep Reinforcement Learning (DRL), has resulted in agents that have achieved superhuman performance in some games. However, DRL can be difficult to apply in practice as it suffers from issues such as sample inefficiency, learning in sparse reward environments, and correct definition of reward functions. The removal of human intervention from the agent's training process has also led to agent behaviour that is unpredictable, uninterpretable, and potentially unsafe. In this work, we use Goal Net, a goal-oriented agent modelling methodology, as a way for agent designers to define an agent's goals and incorporate their prior knowledge about how an agent should achieve goals. As agents become more intelligent, the scenarios in which they can be used will increase, thus increasing the number of potential agent developers and designers. Goal Net uses goals as an abstraction of agent behaviour that can be understood by stakeholders who may have little knowledge about how to implement an agent. Goal Nets can be defined graphically, easing the design process for those who are unfamiliar with programming. We survey recent methods on defining and achieving goals, which include methods related to goal modelling and RL, and identify how the two areas are related. This is followed by an introduction of Goal Net in which we present a method for using Goal Nets for the customization of virtual assistants. Then, we present our method of combining Goal Net and DRL that addresses some of the issues with DRL discussed previously. Experimental results show that our method achieves better results than other methods that incorporate the same level of human knowledge. We then adapt and apply our method to a negotiation dialogue agent. We perform both automatic and human evaluation, and include ChatGPT in the human evaluation as a powerful language generation model to which we can compare. We identify problems with ChatGPT with regards to controllability and usability, and highlight how our proposed method helps mitigate these issues. Finally, we discuss potential future directions for this work and challenges that these directions may pose.