Meta-reinforcement learning in quantitative trading

• Main Author: Wong, Wei Jie
• Other Authors: Bo An
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2022
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence; Business::Information technology::Decision-making
• Online Access: https://hdl.handle.net/10356/156538
• Institution: Nanyang Technological University

Over the past decades, deep reinforcement learning has been a phenomenal artificial intelligence topic in complex games such as Atari video games and Go, due to its superhuman performances in tackling sequential problems. Application of deep reinforcement learning has shown some success in the quantitative trading with different time horizons, from high frequency trading to position trading (Briola, Turiel, Marcaccioli, & Aste, 2021) (Sun, Wang, & An, 2021). However, recent approaches of reinforcement learning in quantitative trading have centred around training an agent on a narrowed task, and often lack the ability to generalize to new variations of the problem. One technique used together with reinforcement learning to train an agent to adapt to new variations of tasks is deep meta-reinforcement learning (Meta-RL) (Wang, et al., 2017). Not only can Meta-RL lead to good generalization performance to a set of similar tasks (Finn, Abbeel, & Levine, 2017) (Nichol, Achiam, & Schulman, 2018), it also reduces the need to train each task from scratch, significantly reducing training time. In this paper, we propose a trading system that uses Model-Agnostic Meta-Learning (MAML) as the Meta-RL algorithm to the quantitative trading problem by training an agent on intra-day trading data of stocks and foreign currency exchange pairs. We will address why Meta-RL is suitable for intra-day quantitative trading and demonstrate that Meta-RL can generate stable positive returns in a non-stationary intra-day trading environment.