Computational techniques for modeling non-player characters in games

• Main Author: Feng, Shu
• Other Authors: Tan Ah Hwee
• Format: Theses and Dissertations
• Language: English
• Published: 2014
• Subjects: DRNTU::Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/61817
• Institution: Nanyang Technological University

Modeling of non-player characters (NPCs) is an important research area in the development of computer games. Autonomous NPCs by emulating the behavior of human beings, with realistic performance and believable affective variations, in simulated environment, make the games more challenging and enjoyable. Modeling of NPCs is essentially the problem of creating autonomous agents, which are expected to function and adapt by themselves in a complex environment. The motivation behind this research is thus to create ``realistic" and ``believable" NPCs with the abilities of autonomy, interactivity, situatedness, learning, and adaptation. Three key problems are considered in this research: (1) how behavior models of NPCs may be learned by mimicking behavior patterns of other layers? (2) how behavior models of NPCs may be adapted through interaction and feedback in a dynamic environment? (3) how emotion of NPCs may be modeled and integrated with the behavior system and to create variations of NPCs? For learning behavior models, this research investigates two classes of self-organizing neural networks. Firstly, the self-generating neural network (SGNN) is investigated to learn behavior rules from specific sample bots in a supervised manner. Further optimization of SGNN is also proposed via a pruning method which improves its performance. Our empirical experiments based on a first person shooting game environment called the Unreal Tournament show that SGNN is able to learn behaviors effectively from their prototype. Secondly, another class of self-organizing neural networks, known as Fusion Architecture for Learning, COgnition, and Navigation (FALCON), is adapted for imitative learning to learn behavior patterns in the Unreal Tournament game. Benchmark experiments are conducted to compare FALCON with SGNN in various aspects. The results show that, compared with SGNN, FALCON is able to achieve a higher level of performance with a much more compact network structure and a much shorter learning time. Moving beyond supervised learning, this research aims to create more versatile NPCs which are able to further learn and adapt during game play in real time. As FALCON is designed to support a myriad of learning paradigms, it is our natural choice for modeling autonomous NPCs in games. Specifically, two hybrid learning strategies, namely the Dual-Stage Learning (DSL) strategy, and the Mixed Model Learning (MML) strategy, are presented to realize the integration of the supervised learning and reinforcement learning in one unified framework. DSL and MML have been applied to creating autonomous non-player characters (NPCs) in the Unreal Tournament game environment. Our experiments show that the NPCs learned with DSL and MML produce a higher level of performance compared with the traditional reinforcement learning and imitative learning. The last but not the least, human factors such as emotion should be considered in NPC modeling to order to develop a ``realistic" and ``believable" agent. Based on the appraisal theory and the theory of cognitive regulated emotion, this research proposes a bio-inspired computational model called Cognitive Regulated Affective Architecture (CRAA) to emulate the structure and computations among neurons and regions of brain for emotion process. CRAA consists of a cognitive network, an appraisal network, and an affective network, which are built and work based on the Adaptive Resonance Theory (ART). Specifically, the cognitive network takes charge of decision making and behavior learning task, whereas the affective network encodes the associations from appraisal components to emotion. In addition, an appraisal network is positioned between the cognitive network and affective network to translate cognitive information to emotion appraisal. The model has also been evaluated in the Unreal Tournament game. Comparing with non-emotional NPC, emotional NPC obtains a higher level of user ratings in focus of game playability and interest.