AI based stock market trending analysis

• Main Author: Chan, Lawann Wen Jun
• Other Authors: Li Fang
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/137950
• Institution: Nanyang Technological University

Forecasting stock market prices is a challenging task that many researchers seek to solve. With the rising trend of using machine learning models to predict stock market prices, creation of an enhanced stock price prediction model is vital in gaining an edge over the market. In this case, social media platforms such as Twitter presents a wealth of information that could facilitate stock market price prediction. Similarly, news publications have been shown to affect the movement trend of stock market prices. Presented in this report is the exploration of sentiments, specifically sentiments from Twitter tweets and The New York Times news articles, in the creation of an enhanced Long Short-Term Memory (LSTM) stock price prediction model. Furthermore, this report compares the performance of a baseline LSTM prediction model without inclusion of sentiments against three sentiment enhanced LSTM prediction model. The extraction of relevant New York Times news articles and Twitter tweets within this report is performed using New York Times API and a Twitter Scrapper. After which, sentiment classification for the extracted news articles is performed using vaderSentiment whereas sentiment classification for the extracted tweets is done using a state-of-the-art natural language processing model, Bidirectional Encoders Representing Transformers (BERT). Additionally, this project introduces a sentiment decay function that incorporates sentiment relevancy to the sentiment scores before input to a LSTM model. Experimental results obtained from the comparison of the baseline LSTM model against the sentiment enhanced LSTM models have demonstrated improvement in price prediction performance for three LSTM models which incorporated sentiment scores. This report concludes that incorporation of sentiments extracted from Twitter tweets and New York Times news articles in the creation of a LSTM stock market price prediction model enhances the model’s performance in stock price prediction. Lastly, this project suggests further avenues for research through the exploration of employing a wider range of sentiments obtained from other news and social media platforms in the generation of a stock price prediction model.