Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications

Technology advancements in wireless communications and high-performance Extended Reality (XR) have empowered the developments of the Metaverse. The demand for the Metaverse applications and hence, real-time digital twinning of real-world scenes is increasing. Nevertheless, the replication of 2D phys...

Full description

Saved in:
Bibliographic Details
Main Authors: Yu, Wenhan, Chua, Terence Jie, Zhao, Jun
Other Authors: School of Computer Science and Engineering
Format: Article
Language:English
Published: 2023
Subjects:
Online Access:https://hdl.handle.net/10356/172579
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-172579
record_format dspace
spelling sg-ntu-dr.10356-1725792023-12-13T06:25:48Z Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications Yu, Wenhan Chua, Terence Jie Zhao, Jun School of Computer Science and Engineering Engineering::Computer science and engineering Reinforcement Learning Wireless Communications Technology advancements in wireless communications and high-performance Extended Reality (XR) have empowered the developments of the Metaverse. The demand for the Metaverse applications and hence, real-time digital twinning of real-world scenes is increasing. Nevertheless, the replication of 2D physical world images into 3D virtual objects is computationally intensive and requires computation offloading. The disparity in transmitted object dimension (2D as opposed to 3D) leads to asymmetric data sizes in uplink (UL) and downlink (DL). To ensure the reliability and low latency of the system, we consider an asynchronous joint UL-DL scenario where in the UL stage, the smaller data size of the physical world images captured by multiple extended reality users (XUs) will be uploaded to the Metaverse Console (MC) to be construed and rendered. In the DL stage, the larger-size 3D virtual objects need to be transmitted back to the XUs. We design a novel multi-agent reinforcement learning algorithm structure, namely Asynchronous Actors Hybrid Critic (AAHC), to optimize the decisions pertaining to computation offloading and channel assignment in the UL stage and optimize the DL transmission power in the DL stage. Extensive experiments demonstrate that compared to proposed baselines, AAHC obtains better solutions with satisfactory training time. Ministry of Education (MOE) Nanyang Technological University This work was supported in part by the Singapore Ministry of Education Academic Research Fund under Grant Tier 1 RG90/22, Grant RG97/20, Grant Tier 1 RG24/20, and Grant Tier 2 MOE2019-T2-1-176; and in part by the Nanyang Technological University-Wallenberg AI, Autonomous Systems and Software Program (WASP) Joint Project. 2023-12-13T06:25:48Z 2023-12-13T06:25:48Z 2023 Journal Article Yu, W., Chua, T. J. & Zhao, J. (2023). Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications. IEEE Journal On Selected Areas in Communications, 41(7), 2138-2157. https://dx.doi.org/10.1109/JSAC.2023.3280988 0733-8716 https://hdl.handle.net/10356/172579 10.1109/JSAC.2023.3280988 2-s2.0-85153893825 7 41 2138 2157 en RG90/22 RG97/20 RG24/20 MOE2019-T2-1-176 IEEE Journal on Selected Areas in Communications © 2023 IEEE. All rights reserved.
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering
Reinforcement Learning
Wireless Communications
spellingShingle Engineering::Computer science and engineering
Reinforcement Learning
Wireless Communications
Yu, Wenhan
Chua, Terence Jie
Zhao, Jun
Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications
description Technology advancements in wireless communications and high-performance Extended Reality (XR) have empowered the developments of the Metaverse. The demand for the Metaverse applications and hence, real-time digital twinning of real-world scenes is increasing. Nevertheless, the replication of 2D physical world images into 3D virtual objects is computationally intensive and requires computation offloading. The disparity in transmitted object dimension (2D as opposed to 3D) leads to asymmetric data sizes in uplink (UL) and downlink (DL). To ensure the reliability and low latency of the system, we consider an asynchronous joint UL-DL scenario where in the UL stage, the smaller data size of the physical world images captured by multiple extended reality users (XUs) will be uploaded to the Metaverse Console (MC) to be construed and rendered. In the DL stage, the larger-size 3D virtual objects need to be transmitted back to the XUs. We design a novel multi-agent reinforcement learning algorithm structure, namely Asynchronous Actors Hybrid Critic (AAHC), to optimize the decisions pertaining to computation offloading and channel assignment in the UL stage and optimize the DL transmission power in the DL stage. Extensive experiments demonstrate that compared to proposed baselines, AAHC obtains better solutions with satisfactory training time.
author2 School of Computer Science and Engineering
author_facet School of Computer Science and Engineering
Yu, Wenhan
Chua, Terence Jie
Zhao, Jun
format Article
author Yu, Wenhan
Chua, Terence Jie
Zhao, Jun
author_sort Yu, Wenhan
title Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications
title_short Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications
title_full Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications
title_fullStr Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications
title_full_unstemmed Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications
title_sort asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications
publishDate 2023
url https://hdl.handle.net/10356/172579
_version_ 1787136589417152512