Neural episodic control with state abstraction
Existing Deep Reinforcement Learning (DRL) algorithms suffer from sample inefficiency.Generally, episodic control-based approaches are solutions that leveragehighly-rewarded past experiences to improve sample efficiency of DRL algorithms.However, previous episodic control-based approaches fail to ut...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | text |
| Language: | English |
| Published: |
Institutional Knowledge at Singapore Management University
2023
|
| Subjects: | |
| Online Access: | https://ink.library.smu.edu.sg/sis_research/8231 https://ink.library.smu.edu.sg/context/sis_research/article/9234/viewcontent/1059_neural_episodic_control_with_s.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Institution: | Singapore Management University |
| Language: | English |
| Summary: | Existing Deep Reinforcement Learning (DRL) algorithms suffer from sample inefficiency.Generally, episodic control-based approaches are solutions that leveragehighly-rewarded past experiences to improve sample efficiency of DRL algorithms.However, previous episodic control-based approaches fail to utilize the latentinformation from the historical behaviors (e.g., state transitions, topological similarities,etc.) and lack scalability during DRL training. This work introducesNeural Episodic Control with State Abstraction (NECSA), a simple but effectivestate abstraction-based episodic control containing a more comprehensive episodicmemory, a novel state evaluation, and a multi-step state analysis. We evaluate ourapproach to the MuJoCo and Atari tasks in OpenAI gym domains. The experimentalresults indicate that NECSA achieves higher sample efficiency than thestate-of-the-art episodic control-based approaches. Our data and code are availableat the project website. |
|---|