AI based stock market trending analysis

• Main Author: Ng, Jun Hao
• Other Authors: Li Fang
• Format: Final Year Project
• Language: English
• Published: 2019
• Subjects: DRNTU::Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: http://hdl.handle.net/10356/76807
• Institution: Nanyang Technological University

In recent years, stock market trending analysis and prediction have become one of the more popular research areas due to the high returns of the stock market. With the dynamic nature of the stock market, various theories such as the Random Walk Theory have highlighted the unpredictability of the stock market, labelling prediction a redundant task. With the advances in Artificial Intelligence (AI) technology, many researches have ventured into the possibilities of using Machine Learning and Deep Learning in stock market prediction. In addition, improving prediction models is still an actively researched area to further enhance the accuracy of stock market prediction. With the rise of social media, huge amount of data is being generated every day and the popularity of incorporating these data into prediction models to enhance the prediction accuracy is rapidly increasing. This project aims to design a hybrid machine learning model that incorporates Sentiment Analysis and Technical Indicators to generate accurate predictions and to discuss the effect of utilizing Sentiment Analysis and Technical Indicators in stock market prediction. For this project, binary classification will be performed on 6 stocks, namely: Dow Jones Industrial Average (DJIA), Google (GOOG), Amazon (AMZN), Apple (AAPL), eBay (EBAY) and Citigroup (C). Sentiment Analysis will be conducted on Top Tweets and New York Times News’ headlines, and 13 Machine Learning Algorithms are considered as the learning-based method for the binary classification. The datasets are split into train set and test set in the ratio of 80:20 and a threshold of 0.005 will be used to determine the stock trend. Observation shows that utilization of Sentiment Analysis and Technical Indicators in the proposed model can generate better accuracy in most cases. And it managed to achieve the highest accuracy of 72.98% when predicting DJIA. As this project only considers Top Tweets and headlines of New York Times News when calculating the daily sentiment values, the effect of the daily sentiment values might be not be maximize. Thus, it is recommended to utilize all possible Tweets and to use the content of News as future enhancement. In addition, optimization of the number of features used can also be performed in attempt to improve the prediction accuracy.