Stock Price Movements Classification Using Machine and Deep Learning Techniques-The Case Study of Indian Stock Market.
24 May 2019-pp 445-452
TL;DR: This paper addresses the technical indicator feature selection and identification of the relevant technical indicators by using Boruta feature selection technique and proposes machine learning techniques and deep learning based model to predict stock price movements.
Abstract: Stock price movements forecasting is an important topic for traders and stock analyst. Timely prediction in stock yields can get more profits and returns. The predicting stock price movement on a daily basis is a difficult task due to more ups and down in the financial market. Therefore, there is a need for a more powerful predictive model to predict the stock prices. Most of the existing work is based on machine learning techniques and considered very few technical indicators to predict the stock prices. In this paper, we have extracted 33 technical indicators based on daily stock price such as open, high, low and close price. This paper addresses the two problems, first is the technical indicator feature selection and identification of the relevant technical indicators by using Boruta feature selection technique. The second is an accurate prediction model for stock price movements. To predict stock price movements we have proposed machine learning techniques and deep learning based model. The performance of the deep learning model is better than the machine learning techniques. The experimental results are significant improves the classification accuracy rate by 5% to 6%. National Stock Exchange, India (NSE) stocks are considered for the experiment.
Citations
More filters
TL;DR: This study empirically proves that a successful prediction performance largely depends on a deliberate combination of feature engineering processes with a baseline learning model to make a good balance and harmony between the curse of dimensionality and the blessing ofdimensionality.
Abstract: The stock market has performed one of the most important functions in a laissez-faire economic system by gathering people, companies, and flows of money for several centuries. There have been numerous studies on the stock market among researchers to predict stock prices, and a growing number of studies employed machine learning or deep learning techniques on the stock market predictions with the advent of big data and the rapid development of artificial intelligence techniques. However, making accurate predictions of stock price direction remains difficult because stock prices are inherently complex, nonlinear, nonstationary, and sometimes too irrational to be predictable. Despite the wealth of information, previous prediction systems often overlooked key indicators and the importance of feature engineering. This study proposes a hybrid GA-XGBoost prediction system with an enhanced feature engineering process consisting of feature set expansion, data preparation, and optimal feature set selection using the hybrid GA-XGBoost algorithm. This study experimentally verifies the importance of feature engineering process in stock price direction prediction by comparing obtained feature sets to original dataset as well as improving prediction performance to outperform benchmark models. Specifically, the most significant accuracy increment comes from feature expansion that adds 67 technical indicators to the original historical stock price data. This study also produces a parsimonious optimal feature set using the GA-XGBoost algorithm that can achieve the desired performance with substantially fewer features. Consequently, this study empirically proves that a successful prediction performance largely depends on a deliberate combination of feature engineering processes with a baseline learning model to make a good balance and harmony between the curse of dimensionality and the blessing of dimensionality.
72 citations
TL;DR: In this article, the authors classified papers according to different deep learning methods, which included: Convolutional neural network (CNN), Long Short-Term Memory (LSTM), Deep Neural Network (DNN), Recurrent Neural Networks (RNN), Reinforcement Learning, and other deep learning method such as HAN, NLP, and Wavenet.
Abstract: The prediction of stock and foreign exchange (Forex) had always been a hot and profitable area of study. Deep learning application had proven to yields better accuracy and return in the field of financial prediction and forecasting. In this survey we selected papers from the DBLP database for comparison and analysis. We classified papers according to different deep learning methods, which included: Convolutional neural network (CNN), Long Short-Term Memory (LSTM), Deep neural network (DNN), Recurrent Neural Network (RNN), Reinforcement Learning, and other deep learning methods such as HAN, NLP, and Wavenet. Furthermore, this paper reviewed the dataset, variable, model, and results of each article. The survey presented the results through the most used performance metrics: RMSE, MAPE, MAE, MSE, accuracy, Sharpe ratio, and return rate. We identified that recent models that combined LSTM with other methods, for example, DNN, are widely researched. Reinforcement learning and other deep learning method yielded great returns and performances. We conclude that in recent years the trend of using deep-learning based method for financial modeling is exponentially rising.
61 citations
TL;DR: Experiments showed that the proposed continuous trend labeling method is a much better state-of-the-art labeling method in terms of classification accuracy and some other classification evaluation metrics, and proved that deep learning models such as LSTM and GRU are more suitable for dealing with the prediction of financial time series data.
Abstract: Time series prediction has been widely applied to the finance industry in applications such as stock market price and commodity price forecasting. Machine learning methods have been widely used in financial time series prediction in recent years. How to label financial time series data to determine the prediction accuracy of machine learning models and subsequently determine final investment returns is a hot topic. Existing labeling methods of financial time series mainly label data by comparing the current data with those of a short time period in the future. However, financial time series data are typically non-linear with obvious short-term randomness. Therefore, these labeling methods have not captured the continuous trend features of financial time series data, leading to a difference between their labeling results and real market trends. In this paper, a new labeling method called "continuous trend labeling" is proposed to address the above problem. In the feature preprocessing stage, this paper proposed a new method that can avoid the problem of look-ahead bias in traditional data standardization or normalization processes. Then, a detailed logical explanation was given, the definition of continuous trend labeling was proposed and also an automatic labeling algorithm was given to extract the continuous trend features of financial time series data. Experiments on the Shanghai Composite Index and Shenzhen Component Index and some stocks of China showed that our labeling method is a much better state-of-the-art labeling method in terms of classification accuracy and some other classification evaluation metrics. The results of the paper also proved that deep learning models such as LSTM and GRU are more suitable for dealing with the prediction of financial time series data.
32 citations
TL;DR: The results show that the convolution neural network and deep learning algorithm can obtain relatively accurate investment strategies, thus ensuring investment returns and reducing investment risks.
28 citations
17 Oct 2019
TL;DR: This work attempts to provide an extensive and objective walkthrough in the direction of applicability of the machine learning algorithms for financial or stock market prediction.
Abstract: Finance is one of the pioneering industries that started using Machine Learning (ML), a subset of Artificial Intelligence (AI) in the early 80s for market prediction. Since then, major firms and hedge funds have adopted machine learning for stock prediction, portfolio optimization, credit lending, stock betting, etc. In this paper, we survey all the different approaches of machine learning that can be incorporated in applied finance. The major motivation behind ML is to draw out the specifics from the available data from different sources and to forecast from it. Different machine learning algorithms has their abilities for predictions and are heavily depended on the number and quality of parameters as input features. This work attempts to provide an extensive and objective walkthrough in the direction of applicability of the machine learning algorithms for financial or stock market prediction.
19 citations
References
More filters
TL;DR: High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
Abstract: The support-vector network is a new learning machine for two-group classification problems. The machine conceptually implements the following idea: input vectors are non-linearly mapped to a very high-dimension feature space. In this feature space a linear decision surface is constructed. Special properties of the decision surface ensures high generalization ability of the learning machine. The idea behind the support-vector network was previously implemented for the restricted case where the training data can be separated without errors. We here extend this result to non-separable training data.
High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated. We also compare the performance of the support-vector network to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
37,861 citations
01 Jul 1992
TL;DR: A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented, applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions.
Abstract: A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented. The technique is applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions. The effective number of parameters is adjusted automatically to match the complexity of the problem. The solution is expressed as a linear combination of supporting patterns. These are the subset of training patterns that are closest to the decision boundary. Bounds on the generalization performance based on the leave-one-out method and the VC-dimension are given. Experimental results on optical character recognition problems demonstrate the good generalization obtained when compared with other learning algorithms.
11,211 citations
TL;DR: This study attempted to develop two efficient models and compared their performances in predicting the direction of movement in the daily Istanbul Stock Exchange (ISE) National 100 Index, finding that average performance of ANN model was found significantly better than that of SVM model.
Abstract: Prediction of stock price index movement is regarded as a challenging task of financial time series prediction. An accurate prediction of stock price movement may yield profits for investors. Due to the complexity of stock market data, development of efficient models for predicting is very difficult. This study attempted to develop two efficient models and compared their performances in predicting the direction of movement in the daily Istanbul Stock Exchange (ISE) National 100 Index. The models are based on two classification techniques, artificial neural networks (ANN) and support vector machines (SVM). Ten technical indicators were selected as inputs of the proposed models. Two comprehensive parameter setting experiments for both models were performed to improve their prediction performances. Experimental results showed that average performance of ANN model (75.74%) was found significantly better than that of SVM model (71.52%).
701 citations
TL;DR: Experimental results show that the performance of all the prediction models improve when these technical parameters are represented as trend deterministic data, and random forest outperforms other three prediction models on overall performance.
Abstract: Four machine learning algorithms are used for prediction in stock markets.Focus is on data pre-processing to improve the prediction accuracy.Technical indicators are discretised by exploiting the inherent opinion.Prediction accuracy of algorithms increases when discrete data is used. This paper addresses problem of predicting direction of movement of stock and stock price index for Indian stock markets. The study compares four prediction models, Artificial Neural Network (ANN), Support Vector Machine (SVM), random forest and naive-Bayes with two approaches for input to these models. The first approach for input data involves computation of ten technical parameters using stock trading data (open, high, low & close prices) while the second approach focuses on representing these technical parameters as trend deterministic data. Accuracy of each of the prediction models for each of the two input approaches is evaluated. Evaluation is carried out on 10years of historical data from 2003 to 2012 of two stocks namely Reliance Industries and Infosys Ltd. and two stock price indices CNX Nifty and S&P Bombay Stock Exchange (BSE) Sensex. The experimental results suggest that for the first approach of input data where ten technical parameters are represented as continuous values, random forest outperforms other three prediction models on overall performance. Experimental results also show that the performance of all the prediction models improve when these technical parameters are represented as trend deterministic data.
657 citations
TL;DR: The experiment shows that SVM by feature extraction using PCA, KPCA or ICA can perform better than that without feature extraction, and among the three methods, there is the best performance in K PCA feature extraction; followed by ICA feature extraction.
524 citations