PRUDEX-Compass: Towards systematic evaluation of reinforcement learning in financial markets

• Main Authors: SUN, Shuo, QIN, Molei, WANG, Xinrun, AN, Bo
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Artificial Intelligence and Robotics; Finance and Financial Management; Theory and Algorithms
• Online Access: https://ink.library.smu.edu.sg/sis_research/9043; https://ink.library.smu.edu.sg/context/sis_research/article/10046/viewcontent/Prudex_pvoa_cc_by.pdf
• Institution: Singapore Management University

The financial markets, which involve more than $90 trillion market capitals, attract the attention of innumerable investors around the world. Recently, reinforcement learning in financial markets (FinRL) has emerged as a promising direction to train agents for making profitable investment decisions. However, the evaluation of most FinRL methods only focuses on profit-related measures and ignores many critical axes, which are far from satisfactory for financial practitioners to deploy these methods into real-world financial markets. Therefore, we introduce PRUDEX-Compass, which has 6 axes, i.e., Profitability, Risk-control, Universality, Diversity, rEliability, and eXplainability, with a total of 17 measures for a systematic evaluation. Specifically, i) since most existing FinRL algorithms are only designed to maximize profit with poor performance under systematic evaluation, we introduce AlphaMix+, which leverages mixture-of-experts and risk-sensitive approaches, to serve as one strong FinRL baseline that outperforms market average on all 6 axes in PRUDEX-Compass, ii) we evaluate AlphaMix+ and 7 other FinRL methods in 4 long-term real-world datasets of influential financial markets to demonstrate the usage of our PRUDEX-Compass and the superiority of AlphaMix+, iii) PRUDEX-Compass1 together with 4 real-world datasets, standard implementation of 8 FinRL methods, a portfolio management environment and related visualization toolkits is released as public resources to facilitate the design and comparison of new FinRL methods. We hope that PRUDEX-Compass can not only shed light on future FinRL research to prevent untrustworthy results from stagnating FinRL into successful industry deployment but also provide a new challenging algorithm evaluation scenario for the reinforcement learning (RL) community.