Fuzzified stock market forecasting with transformer networks and adaptive portfolio management using reinforcement learning

• Main Author: Chia, Samuel Wei Kit
• Other Authors: Quek Hiok Chai
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science
• Online Access: https://hdl.handle.net/10356/181152
• Institution: Nanyang Technological University

This paper presents a novel approach to stock market prediction and portfolio management by integrating deep learning, clustering, and reinforcement learning techniques. This research extends the development of an interpretable Transformer network by [55], which leverages self-attention mechanisms to predict stock prices while providing insights into the importance of historical data points. A key contribution of this paper is the conduction of a large-scale comparison of various clustering algorithms, including K-Means, DBSCAN, and HDBSCAN, to enhance the fuzzification process, enabling better stock groupings and improved prediction accuracy. For portfolio management, we implement the Advantage Actor-Critic (A2C) reinforcement learning model to dynamically optimize asset allocation based on predicted stock prices. We demonstrate enhanced performance in both long-only and long-short strategies by evaluating key financial metrics such as the Sharpe ratio, annualized returns, and maximum drawdown. The model, tested within configurable environments developed using the Gymnasium API, allows for flexible experimentation with portfolio configurations and reward engineering techniques. The results indicate that our model outperforms traditional methods in stock price forecasting and portfolio optimization, delivering enhanced risk-adjusted returns. Future research could explore the integration of multimodal data sources and more advanced reinforcement learning architectures to further improve performance and scalability in real-time trading environments.