Multi-agent reinforcement learning in spatial domain tasks using inter subtask empowerment rewards

In the complex multi-agent tasks, various agents must cooperate to distribute relevant subtasks among each other to achieve joint task objectives. An agent's choice of the relevant subtask changes over time with the changes in the task environment state. Multi-agent Hierarchical Reinforcement L...

Full description

Saved in:
Bibliographic Details
Main Authors: PATERIA, Shubham, SUBAGDJA, Budhitama, TAN, Ah-hwee
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2019
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/6199
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-7202
record_format dspace
spelling sg-smu-ink.sis_research-72022021-09-28T06:24:02Z Multi-agent reinforcement learning in spatial domain tasks using inter subtask empowerment rewards PATERIA, Shubham SUBAGDJA, Budhitama TAN, Ah-hwee In the complex multi-agent tasks, various agents must cooperate to distribute relevant subtasks among each other to achieve joint task objectives. An agent's choice of the relevant subtask changes over time with the changes in the task environment state. Multi-agent Hierarchical Reinforcement Learning (MAHRL) provides an approach for learning to select the subtasks in response to the environment states, by using the joint task rewards to train various agents. When the joint task involves complex inter-agent dependencies, only a subset of agents might be capable of reaching the rewarding task states while other agents take precursory or intermediate roles. The delayed task reward might not be sufficient in such tasks to learn the coordinating policies for various agents. In this paper, we introduce a novel approach of MAHRL called Inter-Subtask Empowerment based Multi-agent Options (ISEMO) in which an Inter-Subtask Empowerment Reward (ISER) is given to an agent which enables the precondition(s) of other agents' subtasks. ISER is given in addition to the domain task reward in order to improve the inter-agent coordination. ISEMO also incorporates options model that can learn parameterized subtask termination functions and relax the limitations posed by hand-crafted termination conditions. Experiments in a spatial Search and Rescue domain show that ISEMO can learn the subtask selection policies of various agents grounded in the inter-dependencies among the agents, as well as learn the subtask termination conditions, and perform better than the standard MAHRL technique. 2019-12-09T08:00:00Z text https://ink.library.smu.edu.sg/sis_research/6199 info:doi/10.1109/SSCI44817.2019.9002777 Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Multi-agent Coordination Reinforcement Learning search and rescue Artificial Intelligence and Robotics Databases and Information Systems
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Multi-agent Coordination
Reinforcement Learning
search and rescue
Artificial Intelligence and Robotics
Databases and Information Systems
spellingShingle Multi-agent Coordination
Reinforcement Learning
search and rescue
Artificial Intelligence and Robotics
Databases and Information Systems
PATERIA, Shubham
SUBAGDJA, Budhitama
TAN, Ah-hwee
Multi-agent reinforcement learning in spatial domain tasks using inter subtask empowerment rewards
description In the complex multi-agent tasks, various agents must cooperate to distribute relevant subtasks among each other to achieve joint task objectives. An agent's choice of the relevant subtask changes over time with the changes in the task environment state. Multi-agent Hierarchical Reinforcement Learning (MAHRL) provides an approach for learning to select the subtasks in response to the environment states, by using the joint task rewards to train various agents. When the joint task involves complex inter-agent dependencies, only a subset of agents might be capable of reaching the rewarding task states while other agents take precursory or intermediate roles. The delayed task reward might not be sufficient in such tasks to learn the coordinating policies for various agents. In this paper, we introduce a novel approach of MAHRL called Inter-Subtask Empowerment based Multi-agent Options (ISEMO) in which an Inter-Subtask Empowerment Reward (ISER) is given to an agent which enables the precondition(s) of other agents' subtasks. ISER is given in addition to the domain task reward in order to improve the inter-agent coordination. ISEMO also incorporates options model that can learn parameterized subtask termination functions and relax the limitations posed by hand-crafted termination conditions. Experiments in a spatial Search and Rescue domain show that ISEMO can learn the subtask selection policies of various agents grounded in the inter-dependencies among the agents, as well as learn the subtask termination conditions, and perform better than the standard MAHRL technique.
format text
author PATERIA, Shubham
SUBAGDJA, Budhitama
TAN, Ah-hwee
author_facet PATERIA, Shubham
SUBAGDJA, Budhitama
TAN, Ah-hwee
author_sort PATERIA, Shubham
title Multi-agent reinforcement learning in spatial domain tasks using inter subtask empowerment rewards
title_short Multi-agent reinforcement learning in spatial domain tasks using inter subtask empowerment rewards
title_full Multi-agent reinforcement learning in spatial domain tasks using inter subtask empowerment rewards
title_fullStr Multi-agent reinforcement learning in spatial domain tasks using inter subtask empowerment rewards
title_full_unstemmed Multi-agent reinforcement learning in spatial domain tasks using inter subtask empowerment rewards
title_sort multi-agent reinforcement learning in spatial domain tasks using inter subtask empowerment rewards
publisher Institutional Knowledge at Singapore Management University
publishDate 2019
url https://ink.library.smu.edu.sg/sis_research/6199
_version_ 1770575846821593088