Autonomous agents in snake game via deep reinforcement learning

Since DeepMind pioneered a deep reinforcement learning (DRL) model to play the Atari games, DRL has become a commonly adopted method to enable the agents to learn complex control policies in various video games. However, similar approaches may still need to be improved when applied to more challengi...

Full description

Saved in:
Bibliographic Details
Main Authors: Wei, Zhepei, Wang, Di, Zhang, Ming, Tan, Ah-Hwee, Miao, Chunyan, Zhou, You
Other Authors: School of Computer Science and Engineering
Format: Conference or Workshop Item
Language:English
Published: 2019
Subjects:
Online Access:https://hdl.handle.net/10356/89882
http://hdl.handle.net/10220/49389
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-89882
record_format dspace
spelling sg-ntu-dr.10356-898822020-03-07T11:48:46Z Autonomous agents in snake game via deep reinforcement learning Wei, Zhepei Wang, Di Zhang, Ming Tan, Ah-Hwee Miao, Chunyan Zhou, You School of Computer Science and Engineering 2018 IEEE International Conference on Agents (ICA) Deep Reinforcement Learning Snake Game Engineering::Computer science and engineering Since DeepMind pioneered a deep reinforcement learning (DRL) model to play the Atari games, DRL has become a commonly adopted method to enable the agents to learn complex control policies in various video games. However, similar approaches may still need to be improved when applied to more challenging scenarios, where reward signals are sparse and delayed. In this paper, we develop a refined DRL model to enable our autonomous agent to play the classical Snake Game, whose constraint gets stricter as the game progresses. Specifically, we employ a convolutional neural network (CNN) trained with a variant of Q-learning. Moreover, we propose a carefully designed reward mechanism to properly train the network, adopt a training gap strategy to temporarily bypass training after the location of the target changes, and introduce a dual experience replay method to categorize different experiences for better training efficacy. The experimental results show that our agent outperforms the baseline model and surpasses human-level performance in terms of playing the Snake Game. NRF (Natl Research Foundation, S’pore) Accepted version 2019-07-17T01:33:46Z 2019-12-06T17:35:45Z 2019-07-17T01:33:46Z 2019-12-06T17:35:45Z 2018-07-01 2018 Conference Paper Wei, Z., Wang, D., Zhang, M., Tan, A.-H., Miao, C., & Zhou, Y. (2018). Autonomous agents in snake game via deep reinforcement learning. 2018 IEEE International Conference on Agents (ICA). doi:10.1109/AGENTS.2018.8460004 https://hdl.handle.net/10356/89882 http://hdl.handle.net/10220/49389 10.1109/AGENTS.2018.8460004 209586 en © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at: https://doi.org/10.1109/AGENTS.2018.8460004 6 p. application/pdf
institution Nanyang Technological University
building NTU Library
country Singapore
collection DR-NTU
language English
topic Deep Reinforcement Learning
Snake Game
Engineering::Computer science and engineering
spellingShingle Deep Reinforcement Learning
Snake Game
Engineering::Computer science and engineering
Wei, Zhepei
Wang, Di
Zhang, Ming
Tan, Ah-Hwee
Miao, Chunyan
Zhou, You
Autonomous agents in snake game via deep reinforcement learning
description Since DeepMind pioneered a deep reinforcement learning (DRL) model to play the Atari games, DRL has become a commonly adopted method to enable the agents to learn complex control policies in various video games. However, similar approaches may still need to be improved when applied to more challenging scenarios, where reward signals are sparse and delayed. In this paper, we develop a refined DRL model to enable our autonomous agent to play the classical Snake Game, whose constraint gets stricter as the game progresses. Specifically, we employ a convolutional neural network (CNN) trained with a variant of Q-learning. Moreover, we propose a carefully designed reward mechanism to properly train the network, adopt a training gap strategy to temporarily bypass training after the location of the target changes, and introduce a dual experience replay method to categorize different experiences for better training efficacy. The experimental results show that our agent outperforms the baseline model and surpasses human-level performance in terms of playing the Snake Game.
author2 School of Computer Science and Engineering
author_facet School of Computer Science and Engineering
Wei, Zhepei
Wang, Di
Zhang, Ming
Tan, Ah-Hwee
Miao, Chunyan
Zhou, You
format Conference or Workshop Item
author Wei, Zhepei
Wang, Di
Zhang, Ming
Tan, Ah-Hwee
Miao, Chunyan
Zhou, You
author_sort Wei, Zhepei
title Autonomous agents in snake game via deep reinforcement learning
title_short Autonomous agents in snake game via deep reinforcement learning
title_full Autonomous agents in snake game via deep reinforcement learning
title_fullStr Autonomous agents in snake game via deep reinforcement learning
title_full_unstemmed Autonomous agents in snake game via deep reinforcement learning
title_sort autonomous agents in snake game via deep reinforcement learning
publishDate 2019
url https://hdl.handle.net/10356/89882
http://hdl.handle.net/10220/49389
_version_ 1681041193970958336