Prediction of Bist Price Indices A Comparative Study Between Traditional and Deep Learning Methods
* Author to whom correspondence should be addressed.
Sigma Journal of Engineering and Natural Sciences 2020, Vol. 38, Issue 4, pp. 1693-1704; doi.org/10.62051/ytu.sigma-journal-of-engineering-and-natural-sciences-prediction-of-bist-price-indices-a-comparative-study-between-traditional-and-dee
Abstract
Keywords: Deep learning; ARIMA; long short-term memory; gated-recurrent unit.
1. Introduction
A time series is a sequence of data points 𝑋 = {𝑥1 , 𝑥2 , … , 𝑥𝑡 } that are measured over time where 𝑥𝑖 represents a single data point at time 𝑡. In many different areas, the real-world data shows the time series behavior and forecasting the future based on past observations becomes an important task in scientific, commercial, industrial or economic decision-making [1]. Especially in finance, this task is noticeably difficult due to the noisy, non-stationary and chaotic nature of financial time series. Mainly two approaches based on statistical and computationally methods have been commonly used [2] in capturing the time series patterns. In financial time series, statistical based methods including Holt-Winters [3], autoregressive integrated moving average (ARIMA) [4], and regression (e.g., linear, support vector and decision tree regressions) have been widely-used as a traditional methods. Due to the statistical properties, ARIMA method is known as the most popular one among them. The method combines both autoregressive (AR) and moving average (MA) processes to construct a composite time series *
Corresponding Author: e-mail: oeyigit@yildiz.edu.tr, tel: (212) 383 44 39
model [5]. Ma ny important studies have been in attempt to predict financial time series using ARIMA models [6-9]. However, ARIMA is in the class of linear models and have some strict assumptions that should be satisfied in the process of constructing models. The non-capability of capturing the non-linear hidden patterns [10] in complex-structured financial time series makes ARIMA prone to the errors. Therefore, some computational techniques which are in the class of non-parametric models have been recently address the challenges of time series prediction and have been preferred instead of traditional methods. Long Short-Term Memory (LSTM) is one of the memory-based architectures in deep learning and was firstly introduced by Hochreiter and Schmidhuber [11]. LSTM is an improved version of recurrent neural network (RNN). One of the main drawback of RNN is the non-capability in storing information over a long time period and its forecasting performance can be easily affected due to the vanishing gradient or exploding gradient. The exploding problem can be relatively easily addressed by enforcing a constraint over the gradient norm [12]. On the other hand, LSTM was designed to overcome the vanishing problem caused in RNN by re-parametrizing RNN. Thus, unlike RNN, LSTM has a capable of learning long-term dependencies. In recent years, the LSTM applications in financial time series data have dramatically increased (see a systematic literature review by [13]). Another improved RNN architecture that is address the problem of vanishing gradient is the Gated-Recurrent Unit (GRU). The main difference between LSTM and GRU architectures is the number of gates. While LSTM consists of three gates namely an input gate, a forget gate and an output gate; GRU has only two gates namely a reset gate and an update gate. Since GRU uses less training parameters, it is advantaged in terms of computational cost and memory usage. On the other hand, LSTM performs better with longer sequence datasets compared to GRU. As in LSTM, GRU method has been becoming one of the flexible and efficient RNN based-architecture used for financial time series [14,15]. Although an extensive literature deal with deep learning in predicting financial time series such as stock market prices or exchange rates at international level, limited studies are available for Turkish stock market. Especially for BIST 30, BIST 50 and BIST 100 price indices, to the best of our knowledge, there has been no previous attempt in comparing the prediction performances of models that are statistical-based and memory-based. This study compares the prediction performances of ARIMA, LSTM and GRU for three BIST price indices. ARIMA model is chosen as a representative of statistical-based prediction method due to the nonstationarity behavior of time series. LSTM and GRU are considered as a representative of memory-based models in deep learning. The rest of this study is organized as follows: Section 2 includes some related works. Section 3 provides detailed descriptions about ARIMA, LSTM and GRU models. Materials including data sets’ collection and descriptions are given in Section 4. The results of each model are presented separately in Chapter 5. Comparisons of models are presented in Section 6. Finally discussion and conclusion regarding the study are presented in Section 7.
3.1. ARIMA model
In ARIMA models, it is assumed that the future value of a data point is a linear function of multiple past data points and random errors which are assumed to follow normal distribution with mean zero and constant variance 𝜎 2 . The model has the following form: 𝒑
where 𝑥𝑡 and 𝜀𝑡 show the observed data point at time 𝑡, respectively. Integers 𝑝, 𝑑 and 𝑞 are the model parameters. ∅𝑖 >0 (𝑖 = 1,2, … , 𝑝) and 𝜃𝑖 > 0 (𝑗 = 1,2, … , 𝑞) in which 𝑝 and 𝑞 denote the orders of autoregressive and moving average processes. To express the model, ARIMA (𝑝, 𝑑, 𝑞) is generally used and 𝑑 shows the order of difference to make the series stationary. When 𝑝 = 0, the model shown with Equation 1 reduces to an MA model with order 𝑞. When 𝑞 = 0, the model becomes an AR model with order 𝑞. If 𝑝 = 𝑞 = 𝑑 = 0, then it means that the process is a white noise. To construct an appropriate ARIMA model which is capable to well-capture the behavior of time series, the estimation of parameters plays an important role. Box-Jenkins methodology presents a practical approach to model the time series with ARIMA [10] and has mainly three steps that are performed by iteratively: (1) the model identification (2) the parameter estimation (3) the diagnostic checking. The model identification step involves the transformation of data in order to achieve stationary in time series. The most commonly used transformation is differencing. According to the stationarity tests such as Augmented Dickey-Fuller (ADF) or sample autocorrelation (ACC) and partial autocorrelation (PACC) functions, it is decided to order of differencing. The computed ACC and PACC of transformed series are adequately describes the appropriate model parameters 𝑝 and 𝑞. With the chosen parameters, model is estimated by maximum likelihood (ML) method and t-test is employed for seeking the statistical significance of each estimated coefficient of parameter. To determine whether the derived model is adequate for forecasting, the diagnostic checking step is performed by handling the residuals generated by
the model. The model is treated as adequate if there are random patterns and independencies among residuals. Otherwise, the entire process is repeated until an adequate model is found.
3.2. LSTM Model
LSTM is a special kind of RNN architecture which was firstly used in language modeling [11]. As in RNN, the network is composed of neurons layers and input is propagated through layers for the purpose of making prediction. Mainly two problems (vanishing gradients and exploding gradients) arise during the RNN training caused by not setting the hyperparameters properly. When the weigh matrix values become so small, the gradient signal gets so small as to have no effect [26]. This leads a non-capable in learning the long-term dependencies in the time series. Thus, a model shows poorer performance in terms of accuracy than it is supposed to be. This is often called as vanishing gradients problem. In addition, if the weigh matrix values become so large, the gradient signal gets so large as to result overflow [25,26], which is called as the problem of exploding gradients. Overcoming the problem of exploding gradients is relatively easier than vanishing gradients problem. Instead of conventional neuron, LSTM uses one or more memory cells which can be controlled by input, output and forget gates [27]. In a LSTM memory cell, the same roles are assigned to input and output gates as in RNN. Unlike RNN, an extra gate, forget gate, is designed in order to decide the discarding information from the cell. Thus, a memory cell is responsible to remember or to forget the current state of the unit that is used or not used in the next time step [28]. Figure 1a shows a simple LSTM architecture. As it is illustrated, LSTM network consists of an input layer, one or more hidden layers and an output layer. The number of neurons in the input layer is as much as the number of explanatory variables in the model. The output space is the number of neurons contained in the output layer. The hidden layer(s) consists of memory cell(s). The input gate in a memory cell (𝑖𝑡 ) decides which information flows into memory cell. When 𝑖𝑡 outputs 0, the blocking is done to prevent flowing information to the next layer. The forget gate (𝑓𝑡 ) decides the discarding information from the cell and outputs a vector that consists of values between 0 and 1. While the 0 output refers that the information is completely discard from the cell, the output of 1 refers that the information is completely remembered. Finally, the output gate (𝑜𝑡 ) is responsible in deciding which information from the cell is used as the output [28,29]. The mathematical expressions of LSTM working are given below: 𝒊𝒕 = 𝝈(𝑾𝒊𝒙 𝒙𝒕 + 𝑾𝒊𝒉 𝒉𝒕−𝟏 + 𝑽𝒊𝒄 𝒄𝒕−𝟏 ) (2) ̌ 𝑪𝒕 = 𝐭𝐚𝐧𝐡(𝑾𝒄𝒙 𝒙𝒕 + 𝑾𝒄𝒉 𝒉𝒕−𝟏 ) (3) 𝒇𝒕 = 𝝈(𝑾𝒇𝒙 𝒙𝒕 + 𝑾𝒇𝒉 𝒉𝒕−𝟏 + 𝑽𝒇𝒄 𝒄𝒕−𝟏 )
where the input of the cell and output of the gate at time 𝑡 are denoted as 𝑥𝑡 and ℎ𝑡 , respectively. The terms 𝑊𝑖𝑥 , 𝑊𝑖ℎ , 𝑊𝑐𝑥 , 𝑊𝑐ℎ , 𝑊𝑓𝑥 , 𝑊𝑓ℎ , 𝑊𝑜𝑥 , 𝑊𝑜ℎ , 𝑉𝑖𝑐 , 𝑉𝑓𝑐 and 𝑉𝑜𝑐 denote corresponding weight matrices. 𝜎 = 1⁄1 + 𝑒 𝑥 is the logistic sigmoid function. ∗ indicates the vectors’ scalar product. By using Equation 2 and Equation 3, the values of input gate and a vector of candidate state values are calculated, respectively. The activation of the forget gate is given with Equation 4. Using the values of 𝑖𝑡 , 𝐶̌𝑡 and 𝑓𝑡 , the value of memory cell is computed as in Equation 5. The output gate value is computed (given in Equation 6) after computation of memory cell state. Finally, the output gate value (ℎ𝑡 ) is found with Equation 7. For the sake of simplicity in formulas, the bias vectors are ignored for 𝑖𝑡 , 𝑓𝑡 , 𝐶𝑡 and 𝑜𝑡 .
1b. Structure of GRU cells Figure 1. Structure of LSTM and GRU NN cells
3.3. GRU model
GRU [30] is the other improved version of RNN to deal with vanishing and exploding gradients problems in RNN. The internal structure of GRU is less complicated than LSTM. Fewer computations in training phase makes GRU faster in computational time as compared with LSTM. Figure 1b shows a simple GRU architecture. GRU network consists of two gates namely a reset (𝑟𝑡 ) and an update (𝑧𝑡 ) and these gates are used in deciding which information will be passed to the output [21]. Specifically, the reset gate decides the amount of information that will be discarded from the cell, whereas the update gate is responsible in deciding the amount of information to be stored in the cell state. The mathematical expressions of GRU working are given below: 𝑧𝑡 = 𝜎(𝑊𝑧𝑥 𝑥𝑡 + 𝑊𝑧ℎ ℎ𝑡−1 ) (8) 𝑟𝑡 = 𝜎(𝑊𝑟𝑥 𝑥𝑡 + 𝑊𝑟ℎ ℎ𝑡−1 )
where the input of the cell and output of the gate at time 𝑡 are denoted as 𝑥𝑡 and ℎ𝑡 , respectively. The terms 𝑊𝑧𝑥 , 𝑊𝑧ℎ , 𝑊𝑟𝑥 , 𝑊𝑟ℎ , 𝑊𝑐𝑥 and 𝑊𝑐ℎ denote corresponding weight matrices. 𝜎 is the logistic sigmoid function and ∗ indicates the vectors’ scalar product. For the sake of simplicity in formulas, the bias vectors are ignored for 𝑧𝑡 , 𝑟𝑡 and 𝐶̌𝑡 .
4.1. Data collection
The sample consists of the historical daily BIST price indices obtained from both The Central Bank of Turkey electronic data distribution system and “Investing.com”. The data sets included BIST 30, BIST 50 and BIST 100 price indices in Borsa İstanbul (BIST). Table 1 shows the start and end dates of the indices available in the mentioned sources and used as data sets in this study. Table 1. Periods of datasets Index Name BIST 30 BIST 50 BIST 100
Starting Date January, 3, 1997 January, 5, 2000 January, 3, 1988
4.2. Data description
Each time series dataset has a set of features namely “Open”, “High”, “Low”, “Close” and “Volume”. Since the features are highly-correlated, the “Close” feature is selected as only the feature of time series to be fed into the ARIMA, LSTM and GRU models. Some descriptive statistics of time series datasets are presented in Table 2. Based on the performed skewness [31] and kurtosis tests [32], the null hypotheses which state that the populations from which the samples are drawn are normally distributed can be rejected for all datasets. This indicates, all of three BIST indices exhibit right-skewed and platykurtic distributions. Table 2. Descriptive statistics of BIST indices BIST Index Name BIST 30 BIST 50 BIST 100
Number of Mean Standard Observations Deviation 5,843 58860.25 41441.98 5,108 52362.64 30003.13 8,236 33798.64 35695.63 *** indicates significant at 1% significance level.
One of the models used in this study is the ARIMA which needs the stationarity of time series data. To check the stationary, two rolling statistics such as mean and standard deviation considering a window size equal to 5 are computed for each data set. Also, ADF tests are performed. In the first panel Figure 2, first, second and third columns shows the rolling statistics calculated for BIST 30, BIST 50 and BIST 100 indices, respectively. At the bottom of each figure, the results of ADF unit root test for each index series are given. As it is shown, calculated tau statistics are less than MacKinnon’s critical values in an absolute term and it can be concluded that the null hypotheses of non-stationary in the time series cannot be rejected at 1%, 5% and 10%
significance levels. To make the time series stationary, first difference is taken and then the rolling statistics and ADF tests are re-calculated (given in the second Panel of Figure 2). As it is demonstrated, after taking the first difference, the null hypothesis of unit root is rejected for each time series dataset at all significance levels. Hence, it can be concluded that all BIST price indices are first-order integrated. Rolling Mean & Standard Deviation for BIST 30
ADF Test Statistic= -21.44 ADF Test Statistic= -20.00 ADF Test Statistic= -23.82 p-value = 0.00 p-value = 0.00 p-value = 0.00 Note: Mac Kinnon’s critical values for 1%, 5% and 10% significance levels are -3.43, -2.86 and 2.56, respectively. Figure 2. Rolling statistics
4.3. Evaluation metric
To compare the prediction performances of models, the root mean square error (RMSE) which is commonly used statistical-based metric is used. RMSE measures the differences between the observed values and values predicted by the model. A relatively high weight is given to large errors [33] and RMSE can be computed as follows: 1
where 𝑁 is the total size of observations, x𝑖 is the observed value and x̂𝑖 is the value predicted by the model. RMSE is the square root of MSE and the model which has the minimum RMSE value is considered as the best one. In evaluating the performance of a constructed model, splitting procedure in which gathered samples are divided into two sub-samples has been commonly used in machine learning applications. In this study, each time series is split into two sub-sets with a ratio of 80:20 according to the study of [20]. The training datasets are used as inputs for training the models, whereas testing datasets are used to evaluate the prediction performances of constructed models.
5.1. ARIMA fitting
The best combination of hyperparameters (𝑝, 𝑑, 𝑞) for the ARIMA models are searched by performing grid method in pre-defined ranges of parameters (𝑝 = [0,4]; 𝑑 = [0,2], 𝑞 = [0,4]). In evaluating the models with different hyperparameters, each is ranked against one another based on how the model is accurately predict the future. The metric used in predicting competitions among ARIMA models is the symmetric mean absolute percentage error (SMAPE). The SMAPE can be defined by: 𝑆𝑀𝐴𝑃𝐸 =
where x𝑖 and x̂𝑖 denote the observed and one-step ahead forecast values, respectively. When a model fit the data perfectly, yet the SMAPE takes the value of 0%. Table 3 shows the best fitted ARIMA model by the grid search for each time series dataset. Table 3. Best fitted ARIMA models Index Name BIST 30 BIST 50 BIST 100
Best Fitted ARIMA(𝑝, 𝑑, 𝑞) ARIMA(2,1,0) ARIMA(0,1,3) ARIMA(3,1,1)
Smape
As it is shown, the best fitted models are ARIMA(2,1,0), ARIMA(0,1,3) and ARIMA(3,1,1) for BIST 30, BIST 50 and BIST 100 indices, respectively. In addition, how well the predictions matched up with the testing datasets for the observed values are shown with Figure 3. BIST 30 Prediction
5.2. LSTM fitting
As in ARIMA, the raw data is needed to be transformed and get ready to be modeled before performing LSTM. Data pre-processing is an important step in machine learning applications to transform the raw data into understandable format. Different methods such as normalization, aggregation and sampling can be used for different purposes (e.g., to accelerate the iteration convergence). Among these pre-processing methods, data normalization is the most commonly used in both classification and regression tasks [34]. This study focuses on min-max normalization (MMN) [35] to re-scale the data sets within the range of 0 to 1. The equation for the MMN is given as follows: 𝑧=
where min(. ) and max(. ) shows the maximum and minimum values of observed values in a dataset 𝑥, respectively. 𝑧 is the normalized value. 1700
Each dataset is transformed by using MMN. LSTM network implementation is performed using Tensorflow library in Python which is an open-source platform designed by Google [36]. In the LSTM network, the number of input layer is set at 1 since each dataset has a “Close” feature. Four hidden layers of LSTM with size of 50 units are used. A sliding window of 60 time steps is used for predictions since the LSTM has the ability of store long term memory state. Thus, the number of previous training set elements for each training set element is 60. Adam optimization algorithm [37] which is an extended version of stochastic gradient descent (SGD) and mean square error (MSE) loss function are used in compiling the LSTM model. Learning rate of the Adam algorithm is set at 0.01. The model is trained on for 50 epochs having 32 batch sizes. Figure 4 illustrates how well the predictions matched up with the testing datasets of BIST 30, BIST 50 and BIST 100 price indices. BIST 30 Prediction
5.3. GRU fitting
When modeling the BIST price indices with GRU, each dataset is transformed by using MMN as in LSTM. Also, the GRU network implementation is performed by Python/Tensorflow library. The number of input layers, hidden layers, time steps, epochs and batch sizes are chosen as the same values as those given in LSTM. Adam algorithm with learning rate 0.01 is used in compiling the GRU model. In addition, MSE loss function is used as the loss function of output layer. Figure 5 illustrates how well the predictions matched up with the testing datasets of BIST 30, BIST 50 and BIST 100 price indices. BIST 30 Prediction
6. Models Comparison Results
Table 4 present the RMSE values for ARIMA, LSTM and GRU models fitting. The last column of Table 4 gives the RMSE-difference between the best and the worst model in terms of RMSE for each BIST index. As it is shown, the traditional approach, ARIMA, gives better results when compared with deep-learning based methods, LSTM and GRU. Among deep learning architectures, while the performance of GRU is better than LSTM for BIST 50 and BIST 100; the LSTM is favored than GRU for BIST 30.
Table 4. Summary of the results BIST Index Name BIST 30 BIST 50 BIST 100
7. Discussion And Conclusion
The stock investment is known as one of the main investment tools in the financial markets. Different indices in Borsa İstanbul have been calculated for a long time in order to be useful guides to the investors who carefully follow market movements. These indices make easier to understand the current status of the market. Also, they are used as benchmarks by the financial industry. In general, investors desire to face minimum risk while making their decisions. However, more risky investment tools might be more profitable since the higher risk may lead to higher profit. Thus, the main research question in making investing decision is how risk can be reduced and how profit can be increased? It is not possible for this question to be answered precisely. However, different traditional and machine learning methods can effectively improve the prediction precision and the outputs generated from these methods can help for investors in avoiding risks and improving benefit [38]. The main purpose of this study is to investigate the performances of traditional and deep learning methods in BIST 30, BIST 50 and BIST 100 price indices prediction. The prediction performance of ARIMA, as a traditional method, is compared against the LSTM and GRU, as deep learning methods. Obtained results show that ARIMA outperforms LSTM and GRU in terms of RMSE. This result may have different causes. First of all, this study employs an ARIMA model for one-step ahead forecasting on univariate datasets. Secondly, the amount of data might be not enough being in the category of "massive". As stated in [39,40], ARIMA generate better performance than deep learning methods specially when the researchers deal with univariate time series. Deep learning methods, on the other hand, have the capability of coping with multivariate time series. Also, deep learning methods deliver on their promise for massive datasets. In the future studies, it is planned to study with massive datasets (e.g., minute-wise index price). In addition, future studies are planned to be focused on extracting various uncorrelated features from the raw data and hence studying with multivariate structure.
Acknowledgements
The authors would like to thank anonymous reviewers for their valuable comments and suggestions to improve the paper.
Share and Cite
YİĞİT, Ö.E.; ALP, S.; ÖZ, E. Prediction of Bist Price Indices A Comparative Study Between Traditional and Deep Learning Methods. Sigma Journal of Engineering and Natural Sciences 2020, Vol. 38, pp. 1693-1704. https://doi.org/10.62051/ytu.sigma-journal-of-engineering-and-natural-sciences-prediction-of-bist-price-indices-a-comparative-study-between-traditional-and-dee