Nonlinearity and high volatility of financial time series have made it difficult to predict stock price. However, thanks to recent developments in deep learning and methods such as long short-term memory (LSTM) and convolutional neural network (CNN) models, significant improvements have been obtained in the analysis of this type of data. Further, empirical mode decomposition (EMD) and complete ensemble empirical mode decomposition (CEEMD) algorithms decomposing time series to different frequency spectra are among the methods that could be effective in analyzing financial time series. Based on these theoretical frameworks, we propose novel hybrid algorithms, i.e., CEEMD-CNN-LSTM and EMD-CNN-LSTM, which could extract deep features and time sequences, which are finally applied to one-step-ahead prediction. The concept of the suggested algorithm is that when combining these models, some collaboration is established between them that could enhance the analytical power of the model. The practical findings confirm this claim and indicate that CNN alongside LSTM and CEEMD or EMD could enhance the prediction accuracy and outperform other counterparts. Further, the suggested algorithm with CEEMD provides better performance compared to EMD.

Stock price prediction using deep learning and frequency decomposition

The stock market has performed one of the most important functions in a laissez-faire economic system by gathering people, companies, and flows of money for several centuries. There have been numerous studies on the stock market among researchers to predict stock prices, and a growing number of studies employed machine learning or deep learning techniques on the stock market predictions with the advent of big data and the rapid development of artificial intelligence techniques. However, making accurate predictions of stock price direction remains difficult because stock prices are inherently complex, nonlinear, nonstationary, and sometimes too irrational to be predictable. Despite the wealth of information, previous prediction systems often overlooked key indicators and the importance of feature engineering. This study proposes a hybrid GA-XGBoost prediction system with an enhanced feature engineering process consisting of feature set expansion, data preparation, and optimal feature set selection using the hybrid GA-XGBoost algorithm. This study experimentally verifies the importance of feature engineering process in stock price direction prediction by comparing obtained feature sets to original dataset as well as improving prediction performance to outperform benchmark models. Specifically, the most significant accuracy increment comes from feature expansion that adds 67 technical indicators to the original historical stock price data. This study also produces a parsimonious optimal feature set using the GA-XGBoost algorithm that can achieve the desired performance with substantially fewer features. Consequently, this study empirically proves that a successful prediction performance largely depends on a deliberate combination of feature engineering processes with a baseline learning model to make a good balance and harmony between the curse of dimensionality and the blessing of dimensionality.

Prediction of stock price direction using a hybrid GA-XGBoost algorithm with a three-stage feature engineering process

The application of Artificial Intelligence (AI) to financial investment is a research area that has attracted extensive research attention since the 1990s, when there was an accelerated technological development and popularization of the personal computer Since then, countless approaches have been proposed to deal with the problem of price prediction in the stock market This paper presents a systematic review of the literature on Artificial Intelligence applied to investments in the stock market based on a sample of 2326 papers from the Scopus website between 1995 and 2019 These papers were divided into four categories: portfolio optimization, stock market prediction using AI, financial sentiment analysis, and combinations involving two or more approaches For each category, the initial introductory research to its state-of-the-art applications are described In addition, an overview of the review leads to the conclusion that this research area is gaining continuous attention and the literature is becoming increasingly specific and thorough

/pdf/artificial-intelligence-applied-to-stock-market-trading-a-3n8oubzmor.pdf

Artificial Intelligence Applied to Stock Market Trading: A Review

Predicting stock market (SM) trends is an issue of great interest among researchers, investors and traders since the successful prediction of SMs' direction may promise various benefits. Because of the fairly nonlinear nature of the historical data, accurate estimation of the SM direction is a rather challenging issue. The aim of this study is to present a novel machine learning (ML) model to forecast the movement of the Borsa Istanbul (BIST) 100 index. Modeling was performed by multilayer perceptron-genetic algorithms (MLP-GA) and multilayer perceptron-particle swarm optimization (MLP-PSO) in two scenarios considering Tanh (x) and the default Gaussian function as the output function. The historical financial time series data utilized in this research is from 1996 to 2020, consisting of nine technical indicators. Results are assessed using Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE) and correlation coefficient values to compare the accuracy and performance of the developed models. Based on the results, the involvement of the Tanh (x) as the output function, improved the accuracy of models compared with the default Gaussian function, significantly. MLP-PSO with population size 125, followed by MLP-GA with population size 50, provided higher accuracy for testing, reporting RMSE of 0.732583 and 0.733063, MAPE of 28.16%, 29.09% and correlation coefficient of 0.694 and 0.695, respectively. According to the results, using the hybrid ML method could successfully improve the prediction accuracy.

Training Multilayer Perceptron with Genetic Algorithms and Particle Swarm Optimization for Modeling Stock Price Index Prediction

Time series prediction has been widely applied to the finance industry in applications such as stock market price and commodity price forecasting. Machine learning methods have been widely used in financial time series prediction in recent years. How to label financial time series data to determine the prediction accuracy of machine learning models and subsequently determine final investment returns is a hot topic. Existing labeling methods of financial time series mainly label data by comparing the current data with those of a short time period in the future. However, financial time series data are typically non-linear with obvious short-term randomness. Therefore, these labeling methods have not captured the continuous trend features of financial time series data, leading to a difference between their labeling results and real market trends. In this paper, a new labeling method called "continuous trend labeling" is proposed to address the above problem. In the feature preprocessing stage, this paper proposed a new method that can avoid the problem of look-ahead bias in traditional data standardization or normalization processes. Then, a detailed logical explanation was given, the definition of continuous trend labeling was proposed and also an automatic labeling algorithm was given to extract the continuous trend features of financial time series data. Experiments on the Shanghai Composite Index and Shenzhen Component Index and some stocks of China showed that our labeling method is a much better state-of-the-art labeling method in terms of classification accuracy and some other classification evaluation metrics. The results of the paper also proved that deep learning models such as LSTM and GRU are more suitable for dealing with the prediction of financial time series data.

https://www.mdpi.com/1099-4300/22/10/1162/pdf

A Labeling Method for Financial Time Series Prediction Based on Trends.

The nature of stock market movement has always been ambiguous for investors because of various influential factors. This study aims to significantly reduce the risk of trend prediction with machine learning and deep learning algorithms. Four stock market groups, namely diversified financials, petroleum, non-metallic minerals and basic metals from Tehran stock exchange, are chosen for experimental evaluations. This study compares nine machine learning models (Decision Tree, Random Forest, Adaptive Boosting (Adaboost), eXtreme Gradient Boosting (XGBoost), Support Vector Classifier (SVC), Naive Bayes, K-Nearest Neighbors (KNN), Logistic Regression and Artificial Neural Network (ANN)) and two powerful deep learning methods (Recurrent Neural Network (RNN) and Long short-term memory (LSTM). Ten technical indicators from ten years of historical data are our input values, and two ways are supposed for employing them. Firstly, calculating the indicators by stock trading values as continuous data, and secondly converting indicators to binary data before using. Each prediction model is evaluated by three metrics based on the input ways. The evaluation results indicate that for the continuous data, RNN and LSTM outperform other prediction models with a considerable difference. Also, results show that in the binary data evaluation, those deep learning methods are the best; however, the difference becomes less because of the noticeable improvement of models' performance in the second way.

/pdf/predicting-stock-market-trends-using-machine-learning-and-50ru6rl37h.pdf

Predicting Stock Market Trends Using Machine Learning and Deep Learning Algorithms Via Continuous and Binary Data; a Comparative Analysis

The prediction of stock groups values has always been attractive and challenging for shareholders due to its inherent dynamics, non-linearity, and complex nature. This paper concentrates on the future prediction of stock market groups. Four groups named diversified financials, petroleum, non-metallic minerals, and basic metals from Tehran stock exchange were chosen for experimental evaluations. Data were collected for the groups based on 10 years of historical records. The value predictions are created for 1, 2, 5, 10, 15, 20, and 30 days in advance. Various machine learning algorithms were utilized for prediction of future values of stock market groups. We employed decision tree, bagging, random forest, adaptive boosting (Adaboost), gradient boosting, and eXtreme gradient boosting (XGBoost), and artificial neural networks (ANN), recurrent neural network (RNN) and long short-term memory (LSTM). Ten technical indicators were selected as the inputs into each of the prediction models. Finally, the results of the predictions were presented for each technique based on four metrics. Among all algorithms used in this paper, LSTM shows more accurate results with the highest model fitting ability. In addition, for tree-based models, there is often an intense competition between Adaboost, Gradient Boosting, and XGBoost.

Deep Learning for Stock Market Prediction

Prediction of stock groups' values has always been attractive and challenging for shareholders. This paper concentrates on the future prediction of stock market groups. Four groups named diversified financials, petroleum, non-metallic minerals and basic metals from Tehran stock exchange are chosen for experimental evaluations. Data are collected for the groups based on ten years of historical records. The values predictions are created for 1, 2, 5, 10, 15, 20 and 30 days in advance. The machine learning algorithms utilized for prediction of future values of stock market groups. We employed Decision Tree, Bagging, Random Forest, Adaptive Boosting (Adaboost), Gradient Boosting and eXtreme Gradient Boosting (XGBoost), and Artificial neural network (ANN), Recurrent Neural Network (RNN) and Long short-term memory (LSTM). Ten technical indicators are selected as the inputs into each of the prediction models. Finally, the result of predictions is presented for each technique based on three metrics. Among all the algorithms used in this paper, LSTM shows more accurate results with the highest model fitting ability. Also, for tree-based models, there is often an intense competition between Adaboost, Gradient Boosting, and XGBoost.

Deep learning for Stock Market Prediction

/pdf/deep-learning-for-stock-market-prediction-1nenrxvbrl.pdf

Hamed Jabani

Papers

Predicting Stock Market Trends Using Machine Learning and Deep Learning Algorithms Via Continuous and Binary Data; a Comparative Analysis

Deep Learning for Stock Market Prediction

Deep learning for Stock Market Prediction

Deep Learning for Stock Market Prediction