An In-Depth Analysis of Cooperative Multi-Robot Hierarchical Reinforcement Learning

Gembong Edhi Setyawan; Pitoyo Hartono; Hideyuki Sawada

doi:10.1145/3568231.3568258

An In-Depth Analysis of Cooperative Multi-Robot Hierarchical Reinforcement Learning

Gembong Edhi Setyawan^*
, Pitoyo Hartono
, Hideyuki Sawada

^*Corresponding author for this work

Faculty of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

In most cases, it is known that Multi-Robot System (MRS) performs more effectively than a Single-Robot System (SRS) in complex environments. Therefore, Deep Reinforcement Learning (DRL) has been gaining interest in training robots to solve tasks in complex environments cooperatively. However, Deep Reinforcement Learning continues to be hampered by challenges in complex environments. Recently, the Multi-Agent Hierarchical Deep Deterministic Policy Gradient (MH-DDPG) has been proposed for implementing hierarchical learning for a group of robots engaged in collaboratively problem-solving. This study provides a more in-depth investigation of the performance of the MH-DDPG algorithm by examining the influence of variations in the number of robots, the sensitivity settings, and the hierarchical levels. The experimental results reveal performance tradeoffs regarding these parameters, which should be considered following the task and environment complexity.

Original language	English
Title of host publication	SIET 2022 - Proceedings of 7th International Conference on Sustainable Information Engineering and Technology 2022
Publisher	Association for Computing Machinery
Pages	119-126
Number of pages	8
ISBN (Electronic)	9781450397117
DOIs	https://doi.org/10.1145/3568231.3568258
Publication status	Published - 22 Nov 2022
Event	7th International Conference on Sustainable Information Engineering and Technology, SIET 2022 - Malang, Indonesia Duration: 22 Nov 2022 → …

Publication series

Name	ACM International Conference Proceeding Series

Conference

Conference	7th International Conference on Sustainable Information Engineering and Technology, SIET 2022
Country/Territory	Indonesia
City	Malang
Period	22/11/22 → …

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 9 Industry, Innovation, and Infrastructure

Keywords

Collaborative robots
Complex environments
Hierarchical deep reinforcement learning
Multi-robot system
Task decomposition

Access to Document

10.1145/3568231.3568258

Cite this

Setyawan, G. E., Hartono, P., & Sawada, H. (2022). An In-Depth Analysis of Cooperative Multi-Robot Hierarchical Reinforcement Learning. In SIET 2022 - Proceedings of 7th International Conference on Sustainable Information Engineering and Technology 2022 (pp. 119-126). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3568231.3568258

@inproceedings{d215cfc5c99c47169f3755bf95bb9f4d,

title = "An In-Depth Analysis of Cooperative Multi-Robot Hierarchical Reinforcement Learning",

abstract = "In most cases, it is known that Multi-Robot System (MRS) performs more effectively than a Single-Robot System (SRS) in complex environments. Therefore, Deep Reinforcement Learning (DRL) has been gaining interest in training robots to solve tasks in complex environments cooperatively. However, Deep Reinforcement Learning continues to be hampered by challenges in complex environments. Recently, the Multi-Agent Hierarchical Deep Deterministic Policy Gradient (MH-DDPG) has been proposed for implementing hierarchical learning for a group of robots engaged in collaboratively problem-solving. This study provides a more in-depth investigation of the performance of the MH-DDPG algorithm by examining the influence of variations in the number of robots, the sensitivity settings, and the hierarchical levels. The experimental results reveal performance tradeoffs regarding these parameters, which should be considered following the task and environment complexity.",

keywords = "Collaborative robots, Complex environments, Hierarchical deep reinforcement learning, Multi-robot system, Task decomposition",

author = "Setyawan, \{Gembong Edhi\} and Pitoyo Hartono and Hideyuki Sawada",

note = "Publisher Copyright: {\textcopyright} 2022 ACM.; 7th International Conference on Sustainable Information Engineering and Technology, SIET 2022 ; Conference date: 22-11-2022",

year = "2022",

month = nov,

day = "22",

doi = "10.1145/3568231.3568258",

language = "English",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

pages = "119--126",

booktitle = "SIET 2022 - Proceedings of 7th International Conference on Sustainable Information Engineering and Technology 2022",

}

Setyawan, GE, Hartono, P & Sawada, H 2022, An In-Depth Analysis of Cooperative Multi-Robot Hierarchical Reinforcement Learning. in SIET 2022 - Proceedings of 7th International Conference on Sustainable Information Engineering and Technology 2022. ACM International Conference Proceeding Series, Association for Computing Machinery, pp. 119-126, 7th International Conference on Sustainable Information Engineering and Technology, SIET 2022, Malang, Indonesia, 22/11/22. https://doi.org/10.1145/3568231.3568258

An In-Depth Analysis of Cooperative Multi-Robot Hierarchical Reinforcement Learning. / Setyawan, Gembong Edhi; Hartono, Pitoyo; Sawada, Hideyuki.
SIET 2022 - Proceedings of 7th International Conference on Sustainable Information Engineering and Technology 2022. Association for Computing Machinery, 2022. p. 119-126 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - An In-Depth Analysis of Cooperative Multi-Robot Hierarchical Reinforcement Learning

AU - Setyawan, Gembong Edhi

AU - Hartono, Pitoyo

AU - Sawada, Hideyuki

PY - 2022/11/22

Y1 - 2022/11/22

N2 - In most cases, it is known that Multi-Robot System (MRS) performs more effectively than a Single-Robot System (SRS) in complex environments. Therefore, Deep Reinforcement Learning (DRL) has been gaining interest in training robots to solve tasks in complex environments cooperatively. However, Deep Reinforcement Learning continues to be hampered by challenges in complex environments. Recently, the Multi-Agent Hierarchical Deep Deterministic Policy Gradient (MH-DDPG) has been proposed for implementing hierarchical learning for a group of robots engaged in collaboratively problem-solving. This study provides a more in-depth investigation of the performance of the MH-DDPG algorithm by examining the influence of variations in the number of robots, the sensitivity settings, and the hierarchical levels. The experimental results reveal performance tradeoffs regarding these parameters, which should be considered following the task and environment complexity.

AB - In most cases, it is known that Multi-Robot System (MRS) performs more effectively than a Single-Robot System (SRS) in complex environments. Therefore, Deep Reinforcement Learning (DRL) has been gaining interest in training robots to solve tasks in complex environments cooperatively. However, Deep Reinforcement Learning continues to be hampered by challenges in complex environments. Recently, the Multi-Agent Hierarchical Deep Deterministic Policy Gradient (MH-DDPG) has been proposed for implementing hierarchical learning for a group of robots engaged in collaboratively problem-solving. This study provides a more in-depth investigation of the performance of the MH-DDPG algorithm by examining the influence of variations in the number of robots, the sensitivity settings, and the hierarchical levels. The experimental results reveal performance tradeoffs regarding these parameters, which should be considered following the task and environment complexity.

KW - Collaborative robots

KW - Complex environments

KW - Hierarchical deep reinforcement learning

KW - Multi-robot system

KW - Task decomposition

UR - https://www.scopus.com/pages/publications/85146944463

U2 - 10.1145/3568231.3568258

DO - 10.1145/3568231.3568258

M3 - Conference contribution

AN - SCOPUS:85146944463

T3 - ACM International Conference Proceeding Series

SP - 119

EP - 126

BT - SIET 2022 - Proceedings of 7th International Conference on Sustainable Information Engineering and Technology 2022

PB - Association for Computing Machinery

T2 - 7th International Conference on Sustainable Information Engineering and Technology, SIET 2022

Y2 - 22 November 2022

ER -

Setyawan GE, Hartono P, Sawada H. An In-Depth Analysis of Cooperative Multi-Robot Hierarchical Reinforcement Learning. In SIET 2022 - Proceedings of 7th International Conference on Sustainable Information Engineering and Technology 2022. Association for Computing Machinery. 2022. p. 119-126. (ACM International Conference Proceeding Series). doi: 10.1145/3568231.3568258

An In-Depth Analysis of Cooperative Multi-Robot Hierarchical Reinforcement Learning

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this