Transforming cooling optimization for green data center via deep reinforcement learning

Data center (DC) plays an important role to support services, such as e-commerce and cloud computing. The resulting energy consumption from this growing market has drawn significant attention, and noticeably almost half of the energy cost is used to cool the DC to a particular temperature. It is thu...

Full description

Saved in:
Bibliographic Details
Main Authors: Li, Yuanlong, Wen, Yonggang, Tao, Dacheng, Guan, Kyle
Other Authors: School of Computer Science and Engineering
Format: Article
Language:English
Published: 2021
Subjects:
Online Access:https://hdl.handle.net/10356/154224
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-154224
record_format dspace
spelling sg-ntu-dr.10356-1542242021-12-31T13:44:13Z Transforming cooling optimization for green data center via deep reinforcement learning Li, Yuanlong Wen, Yonggang Tao, Dacheng Guan, Kyle School of Computer Science and Engineering Engineering::Computer science and engineering Data Center (DC) Cooling Optimization , Deep Learning Data center (DC) plays an important role to support services, such as e-commerce and cloud computing. The resulting energy consumption from this growing market has drawn significant attention, and noticeably almost half of the energy cost is used to cool the DC to a particular temperature. It is thus an critical operational challenge to curb the cooling energy cost without sacrificing the thermal safety of a DC. The existing solutions typically follow a two-step approach, in which the system is first modeled based on expert knowledge and, thus, the operational actions are determined with heuristics and/or best practices. These approaches are often hard to generalize and might result in suboptimal performances due to intrinsic model errors for large-scale systems. In this paper, we propose optimizing the DC cooling control via the emerging deep reinforcement learning (DRL) framework. Compared to the existing approaches, our solution lends itself an end-to-end cooling control algorithm (CCA) via an off-policy offline version of the deep deterministic policy gradient (DDPG) algorithm, in which an evaluation network is trained to predict the DC energy cost along with resulting cooling effects, and a policy network is trained to gauge optimized control settings. Moreover, we introduce a de-underestimation (DUE) validation mechanism for the critic network to reduce the potential underestimation of the risk caused by neural approximation. Our proposed algorithm is evaluated on an EnergyPlus simulation platform and on a real data trace collected from the National Super Computing Centre (NSCC) of Singapore. The resulting numerical results show that the proposed CCA can achieve up to 11% cooling cost reduction on the simulation platform compared with a manually configured baseline control algorithm. In the trace-based study of conservative nature, the proposed algorithm can achieve about 15% cooling energy savings on the NSCC data trace. Our pioneering approach can shed new light on the application of DRL to optimize and automate DC operations and management, potentially revolutionizing digital infrastructure management with intelligence. date of current version April 15, 2020. This work was supported in part by the Green Data Centre Research Project, administrated by the Singapore Infocomm and Media Development Authority. This paper was recommended by Associate Editor Y. Zhang 2021-12-16T03:47:59Z 2021-12-16T03:47:59Z 2020 Journal Article Li, Y., Wen, Y., Tao, D. & Guan, K. (2020). Transforming cooling optimization for green data center via deep reinforcement learning. IEEE Transactions On Cybernetics, 50(5), 2002-2013. https://dx.doi.org/10.1109/TCYB.2019.2927410 2168-2267 https://hdl.handle.net/10356/154224 10.1109/TCYB.2019.2927410 31352360 2-s2.0-85071908443 5 50 2002 2013 en IEEE Transactions on Cybernetics © 2019 IEEE. All rights reserved.
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering
Data Center (DC) Cooling Optimization ,
Deep Learning
spellingShingle Engineering::Computer science and engineering
Data Center (DC) Cooling Optimization ,
Deep Learning
Li, Yuanlong
Wen, Yonggang
Tao, Dacheng
Guan, Kyle
Transforming cooling optimization for green data center via deep reinforcement learning
description Data center (DC) plays an important role to support services, such as e-commerce and cloud computing. The resulting energy consumption from this growing market has drawn significant attention, and noticeably almost half of the energy cost is used to cool the DC to a particular temperature. It is thus an critical operational challenge to curb the cooling energy cost without sacrificing the thermal safety of a DC. The existing solutions typically follow a two-step approach, in which the system is first modeled based on expert knowledge and, thus, the operational actions are determined with heuristics and/or best practices. These approaches are often hard to generalize and might result in suboptimal performances due to intrinsic model errors for large-scale systems. In this paper, we propose optimizing the DC cooling control via the emerging deep reinforcement learning (DRL) framework. Compared to the existing approaches, our solution lends itself an end-to-end cooling control algorithm (CCA) via an off-policy offline version of the deep deterministic policy gradient (DDPG) algorithm, in which an evaluation network is trained to predict the DC energy cost along with resulting cooling effects, and a policy network is trained to gauge optimized control settings. Moreover, we introduce a de-underestimation (DUE) validation mechanism for the critic network to reduce the potential underestimation of the risk caused by neural approximation. Our proposed algorithm is evaluated on an EnergyPlus simulation platform and on a real data trace collected from the National Super Computing Centre (NSCC) of Singapore. The resulting numerical results show that the proposed CCA can achieve up to 11% cooling cost reduction on the simulation platform compared with a manually configured baseline control algorithm. In the trace-based study of conservative nature, the proposed algorithm can achieve about 15% cooling energy savings on the NSCC data trace. Our pioneering approach can shed new light on the application of DRL to optimize and automate DC operations and management, potentially revolutionizing digital infrastructure management with intelligence.
author2 School of Computer Science and Engineering
author_facet School of Computer Science and Engineering
Li, Yuanlong
Wen, Yonggang
Tao, Dacheng
Guan, Kyle
format Article
author Li, Yuanlong
Wen, Yonggang
Tao, Dacheng
Guan, Kyle
author_sort Li, Yuanlong
title Transforming cooling optimization for green data center via deep reinforcement learning
title_short Transforming cooling optimization for green data center via deep reinforcement learning
title_full Transforming cooling optimization for green data center via deep reinforcement learning
title_fullStr Transforming cooling optimization for green data center via deep reinforcement learning
title_full_unstemmed Transforming cooling optimization for green data center via deep reinforcement learning
title_sort transforming cooling optimization for green data center via deep reinforcement learning
publishDate 2021
url https://hdl.handle.net/10356/154224
_version_ 1722355367488258048