DRAM failure prediction in AIOps: Empirical evaluation, challenges and opportunities

• Main Authors: WU, Zhiyue, XU, Hongzuo, PANG, Guansong, YU, Fengyuan, WANG, Yijie, JIAN, Songlei, WANG, Yongjun
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2021
• Subjects: DRAM failure prediction; Data center reliability; Cloud services; Databases and Information Systems; Data Storage Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/7135; https://ink.library.smu.edu.sg/context/sis_research/article/8138/viewcontent/2104.15052.pdf
• Institution: Singapore Management University

DRAM failure prediction is a vital task in AIOps, which is crucial to maintain the reliability and sustainable service of large-scale data centers. However, limited work has been done on DRAM failure prediction mainly due to the lack of public available datasets. This paper presents a comprehensive empirical evaluation of diverse machine learning techniques for DRAM failure prediction using a large-scale multisource dataset, including more than three millions of records of kernel, address, and mcelog data, provided by Alibaba Cloud through PAKDD 2021 competition. Particularly, we first formulate the problem as a multiclass classification task and exhaustively evaluate seven popular/stateof-the-art classifiers on both the individual and multiple data sources. We then formulate the problem as an unsupervised anomaly detection task and evaluate three state-of-the-art anomaly detectors. Further, based on the empirical results and our experience of attending this competition, we discuss major challenges and present future research opportunities in this task.