Artificial Neural Networks Restriction for Road Accidents Severity Classification in Unbalanced Data

Abstract

Keywords: Unbalanced data; road accidents; severity; classification; artificial neural networks.

1. Introduction

In order to carry out measures of prevention and reduction of road accident severity, it is necessary to perform an analysis of the occurrence of road accidents. Such analysis would include associating the record of each accident occurrence with the characteristics of the road, the environment, the vehicular conditions and the driver. These characteristics, when related to accident records, can be used in the prediction and classification of road accident severity [1]. Several methods about road accident analysis can be found in the literature. The principal methods are based on deterministic models and regression analysis, such as logistic regression, negative binominal regression and Poisson regression, which investigate the individual contribution of a specific factor in the variation of road accident severity [2, 3]. Al-Ghamdi [4] *

Corresponding Author: e-mail: marialigia@ufu.br, tel: +55 34 3239-4411

applied the logistic regression to analyze the injury severity in 560 accident occurrences involving severe (fatal and non-fatal) victims, in Riyadh, the capital of Saudi Arabia. Farmer et al. [5] used logistic regression to analyze the injury severity in shock type accidents involving passenger vehicles and light trucks in the United States for a period of 4 years (1988 to 1992). Lui et al. [6] quantified through logistic regression the relationships between the use of safety belts, the direction of impacts in crash accidents, and the weight of vehicles, and the effects of these factors on accidents involving fatalities. Singleton, Qin & Luan [7] used logistic regression to identify driver, vehicle and pathway/environment factors associated with the increased risk of serious injury in Kentucky from 2000 to 2001. Negative binominal regression was used by Pirdavani, Brijs & Bellemans [8] to assess the impact of traffic safety on the basis of fuel price increases for the period 2004-2007 in Flanders (Belgium) in 2,200 traffic zones. Poisson regression was used by Ye et al. [9] in the generation of accident severity frequency rate models for highway sections using five years of data for 275 multilane road segments in Washington State. Debrabant et al. [10] used the same regression model for the identification of critical areas of road accidents in Funen, Denmark, for the period from 2002 to 2007. Regression models provide analytical results, based on a predetermined mathematical function. If the function requirements are violated, erroneous probabilities are obtained for accident severity. However, traffic accidents are multicausal events of random nature and cannot be described by simple deterministic models [2]. Therefore, probability models that describe the random behavior of the entities present in the accident database should be used, in order to convey a greater number of parameters associated to the road accident phenomena [2]. In this perspective, stochastic and non-parametric approaches, derived from artificial intelligence, have been employed in the treatment of heterogeneous and multidimensional databases [2, 3, 11, 12, 13, 14]. Among them are those based on data mining, in which there is no need to establish a priori restrictions on the relationships of the database variables. Data mining techniques used in investigating accident severity fundamentally use tree structures and networks. The techniques based on tree structures, known as decision trees, restrict the results obtained due to the hierarchical and binary structure of the tree. This feature limits the application of tree structure techniques to the analysis of specific categories of a target variable (dependent) belonging to the database, being ineffective for the modelling of multiple problems [11, 15,16]. Data mining techniques based on network structures allow a greater extraction of knowledge from the database. Since a larger number of connections can be made between different types of data, the modelling process can be more flexible [2, 17]. Other studies in this area highlight the use of Artificial Neural Networks (ANN) [3, 12, 18, 19, 20, 21] and Bayesian networks [22, 23, 24, 25] when dealing with non-linear problems in which data do not present a normal distribution [26].The results obtained in the previous studies depict the effectiveness of network modelling in the treatment of large and heterogeneous databases, with emphasis on the prediction and classification process [2]. Considering that ANN have been successfully employed in non-linear problems with a nonnormal distribution, the use of ANN in the evaluation of accident database, which has such characteristics, is a feasible hypothesis. However, some limitations emerge regarding the severity classification in unbalanced databases, where fatal and severe accidents are underrepresented in the total number of accident occurrences. Based on this problem, the scope of this research is to present a discussion about the restriction of ANN for road accident severity classification, using unbalanced traffic accidents database. The most significant variables were selected, in order to correlate the degree of injury with the main risk factors in the road environment. This paper is organized as follows. Section 2 presents the materials and method adopted in the research, with a brief description of variable selection and learning based classification on ANN with multiple layers. Section 3 presents the experiments and discusses the results obtained.

Section 4 presen ts the conclusions about the approach employed, as well as the recommendations for future work.

1. N K

where t and a are the observed and estimated parameters, respectively; K is the number of neurons of the output layer and N the number of test set data (Chang, 2005). In this work, the SPSS software was used to construct ANN with Multiple Layers Perceptron (MLP), based on the "back propagation” algorithm, which corrects the weights in all layers, starting from the output layer to the input layer, using the mean square root error, in the two phases: the phase forward and the phase for backward. In the phase forward, each variable of database is stored in a neuron of the network. Na fase forward, cada variável da base de dados é armazenada em um neurônio da rede. A ponderação inicial se dá pelas conexões ou relações existentes entre as variáveis da camada de entrada. Cada neurônio nessa camada aplica a função de ativação a sua entrada total e produz um valor de saída, que é utilizado como valor de entrada pelos neurônios da camada seguinte. Esse processo se dá iterativamente até que os neurônios da camada de saída produzam cada um seu valor de saída, que é então comparado ao valor desejado para a saída desse neurônio. A diferença entre os valores produzidos e desejados para cada

neurônio da camada de saída expressa o erro do processo de modelagem dos dados (Facelli et al. 2011). O valor do erro de cada neurônio da camada de saída é então utilizado na fase backward, no ajuste dos pesos de entrada. O ajuste ocorre da camada de saída até a camada intermediária. O ajuste dos pesos de uma MLP pelo algoritmo back-propagation pode ser obtido pela seguinte equação (Facelli et al. 2011):

onde, w j ,l representa o peso entre um neurônio l e o j  ésimo atributo de entrada ou saída do j  ésimo neurônio da camada anterior;  l o erro associado ao l  ésimo neurônio; x

corresponde a entrada recebida por esse neurônio, ou seja, o j  ésimo atributo de entrada ou a saída j  ésimo neurônio da camada anterior; e  a taxa de aprendizado da modelagem. Nesta modelagem apenas os valores dos erros dos neurônios de saída são conhecidos. Desta forma, deve-se estimar o erro para as camadas intermediárias. No algoritmo back-propagation, o erro de um neurônio de uma camada intermediária é estimado como a soma dos erros dos neurônios da camada seguinte, cujos terminais de entrada estão conectados a ele. A ponderação ocorre pelo peso atribuído a estas conexões. Deste modo, o cálculo do erro dependerá da camada em que se encontra o neurônio, como mostra a equação (3) e a equação (4), (Facelli et al. 2011):

onde, nl representa o l  ésimo neurônio; f a' a derivada parcial da função de ativação do neurônio; e el o erro quadrático cometido pelo neurônio de saída quando sua resposta ( yq ) é comparada a desejada ( f q ) , definido por:

O ajuste dos pesos é definido pela f a' , utilizando o gradiente descente da função de ativação. Essa derivada mede a contribuição de cada peso no erro da rede para cada variável utilizada na análise. Quando o valor da f a' é positivo para um dado peso, isto indica que o peso atribuído a uma determinada variável está provocando um aumento da diferença entre os valores obtidos e desejados na modelagem. Neste caso, a magnitude do peso deve ser reduzida para minimizar o erro obtido. E, caso contrário, quando o valor da f a' for negativo, o peso deve ser aumentado, para que os valores obtidos se aproximem dos valores esperados (Facelli et al. 2011). Figure 3 shows the structure of the ANN with MLP used in the modelling of the database.

Figure 3. Illustration of ANN model with MLP. The first subset uses 70% of the data for ANN training and the second subset uses 30% of the data for the ANN test. These subsets are used iteratively for learning, validation and crossvalidation without repetition. At the end of the processing, the whole set of data was verified. These subsets should not be too large, because they may hinder the ANN learning process. In addition, training and test subsets should be as homogeneous as possible for each class of variables, since ANN learning is based on as many combinations as possible. When many variables are used, or the data are not homogeneously distributed, the quality of ANN learning also tends to decline. Therefore, in the input layer of ANN a small number of variables must be selected, in order to obtain best results both in data extraction and in class distribution for the test and training subsets. In addition, the values of the input variables (predictors) must be normalized. In this study, the input layer presents 39 neurons. The variable type of accident is represented by 10 neurons; the variable weather condition by 5 neurons; the variable accident cause by 4 neurons and the variable mileage by 1 neuron. The other variables, such as visibility, road profile, road geometry, pavement condition, and period are represented with 3 neurons each. Horizontal signal and vertical signal are represented with 2 neurons each. The hidden layer contains the nodes or unobserved units and, in this study, has 9 neurons. The output layer contains two response neurons (accidents without victims and accidents with fatal and non-fatal victims). Therefore, each neuron belonging to the output layer represents a predicted variable, whereas the hidden layer neurons are directly connected to the values of the independent variables. For the transfer of information between the input-layer neurons and the hidden layer, the hyperbolic tangent activation function was used, when the selection of the network architecture was performed automatically. In addition, for the transfer of information between the hidden layer and the output layer, the Softmax activation function was used, when all the independent variables are categorical. The exact relationships between input and output data are not known, which implies that the optimal number of cases is not known for learning and for setting up the best ANN model. The number of cases used in the network learning process was 2,824 accidents, of which 1,964 (69.5%) were used as a training subset and 860 (30.5%) as a test subset. 932

3. Results And Discussion

Table 1 shows the amount of data classified following the ANN model with MPL for each category of injury accidents being accidents with no injury (NI) and accidents with injury (WI). Table 1. Classification of accidents in relation to victims. Sub-sets Training

WI 460 18 (%) Total 100.00% 0.00% NI 646 8 WI 182 14 (%) Total 97.4% 2.6% *no injury (NI) and with injury (WI).

As shown in Table 1 and in Figure 4, a two-class model (NI and WI) resulted in a general prediction of 77.9%. The results obtained in the classification process – using the ANN and the eleven predictor variables and the variables NI and WI.

Figure 4. Quality of the model fit for the dependent variable “accident severity”. However, ANN ranked fatalities (WI) with an accuracy of only 2.6%, while non-fatal accidents (NI) were classified with high accuracy (97.4%). This is a typical particularity of multiple variable problems, in which the database is naturally unbalanced, in relation to the number of observations and the number of variable categories under analysis [19]. Figure 5 shows the Receiver Operating Characteristic (ROC) curve and Figure 6 shows the gain for each dependent variable (NI and WI) as a function of the ROC curve.

Figure 5. ROC curve obtained for the dependent variable “accident severity”.

Figure 6. Gain graph for the dependent variable “accident severity”. The ROC curve corresponds to the graphical representation of sensitivity (true positives) versus specificity (false positives), ranging from zero to one, and provides a measure of the discrimination of the model. When analyzing the ROC curve (Figure 5), the values predicted for the classes of the severity variable NI and WI should be very close to the diagonal, that is, close to one. However, the classification presented mean values of 0.618 for both NI predicted accidents and WI predicted accidents. Figure 6 shows the graph of cumulative gains for categorical variables NI and WI accidents, as a function of the ROC curve. Note that the gain balance between the two variables occurs when there is the best model fit (precision equal to 77.9%). For this method, the variable accident type was the most important in the prediction process, followed by the variables accident cause, vertical signal, visibility and mileage. The other variables used in the modelling presented importance below 40%, as can be seen in Table 2. The

percentage of uncorrected predictions for the training subset was 25.2% and for the test subset was 22.1%. Table 2. Importance of independent variables. Variables ACT ACC PER SGC WTC PAV GER PFR HS VS km

Standardized Importance 100.0% 65.6% 12.6% 44.0% 35.8% 33.1% 23.5% 28.9% 16.3% 46.6% 42.7%

The Figure 7 shows that the four most important variables accident type, period, visibility, weather condition, road geometry, road profile, pavement condition, milestone, horizontal signal and vertical signal have a nonhomogeneous data distribution.

Figure 7. Nonhomogeneous data distribution of the main variables.

Figures 8 and 9 present, respectively, the scores obtained for the predictions of NI and WI accidents.

Figure 8. Scores obtained in the prediction of accidents with no injury (NI).

Figure 9. Scores obtained in the prediction of accidents with injury (WI). The classification for NI accidents resulted in an average score of 0.76, with minimum value of 0.32 and maximum value of 0.89. While the classification for WI accidents resulted in an average score of 0.24, with minimum value of 0.11 and maximum value of 0.68. Therefore, in general terms, the prediction and classification of NI accidents presented the highest scores and, consequently, the highest probabilities of accuracy. According to Hosmer and Lemeshow [27], values of 0.5  AUC  0.7 provide nondiscriminatory modeling. Values in the range of 0.7  AUC  0.8 provide a model with acceptable discrimination, whereas values between 0.8  AUC  0.9 indicate excellent modeling and AUC  0.9 a model with extraordinary discrimination potential. Thus, considering the limitations of ANN for unbalanced data [13, 19] and the predictions that demonstrated larger evidence or probability of occurrence, it was adopted an AUC  0.6 , aiming only at the exploitation of the results obtained. Table 3 shows the means and standard 936

deviations for the predictive variables, considering the severity dependent variable of NI and WI accidents. Table 3. Performance of the accident severity classification. Variable ACT

2. Not present

NI Mean (%) 81.43 18.57 0.00 90.12 5.41 4.48 27.36 45.39 27.24 99.19 0.81 99.44 0.56

S. D. (%) 6.75 6.75 0.00 8.38 5.14 4.55 9.26 7.93 3.68 0.65 0.60 0.63 0.55

WI Mean (%) S. D. (%) 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 25.00 7.50 25.00 7.50 16.67 5.00

WI Mean (%) S. D. (%) 71.88 5.94 0.00 0.00 28.13 5.94 50.00 5.00 0.00 0.00 50.00 5.00 30.21 7.76 0.00 0.00 69.79 7.76 100.00 0.00 0.00 0.00 100.00 0.00 0.00 0.00

Considering that the AUC is numerically equal to the probability [28], it is seen that the accidents most likely not to cause victims (NI) are the rear-end collision (29.98%), lateral collision (15.09%), pile-up (13.57%) and crash with fixed or mobile object (36.24%). Most of these accidents have as probable cause driver behaviour (71.38%); happen most likely in the morning (38.02%) and in the afternoon (43.28%); with good visibility (62.93%) or partial visibility (36.09%); in good weather condition (80.86%); dry pavement condition (81.43%); in straight segments (90.12%); in level profile (45.39%); and in segments where there is horizontal (99.19%) and vertical (99.44%) signals. The accidents most likely to cause fatalities (WI) are rollover type (25%), pedestrian collision (25%) and fall of cyclists and motorcyclist (33.33%). Most of these accidents have as probable cause road and environment factor (59.38%); happen most likely in the afternoon (59.38%); with a normal (46.88%) or partial (53.13%) visibility condition; good weather condition (93.75%); dry pavement condition (71.88%); in straight segments (50%) or sharp curve (50%); with descending (30.21%) or ascending (69.79%) road profile; and in segments where there is horizontal (100%) and vertical (100%) signals.

4. Conclusions

This article presented a discussion of the main limitations encountered when using the ANN approach for road accident severity classification based on unbalanced databases, considering eleven variables that encompass road infrastructure, environment and type of accidents. The results obtained with the ANN had accuracy of 77.9% and with mean AUC values of 0.618, compatible with the values found in the literature [13,19]. However, an accident rate accuracy of only 2.6% for WI and a high accident rate accuracy for NI (97.4%) were obtained, as observations associated with accidents with victims were more unusual in the data set of traffic accident records (unbalanced dataset). Because of this limitation, some authors have been exploring data balancing techniques and used series of binary classifiers in ANN modelling, aiming to reduce the training time of the model and increase the accuracy of the general prediction. In this way, it is possible to use resources such as weights of connections in networks, in order to simplify the structure of the model for a better generalization of the results obtained [13, 29]. The ANN architecture allows reconstructing any continuous function independent of the input data sequence, when the iterative learning process uses the entire data set and the actual associations between the categories of the selected variables and the target variable. Thus, the lower the number of predictor variables used, the greater the effectiveness of results generalization obtained in ANN modelling. Nevertheless, although the ANN is a robust tool for solving complex, multiple-class problems, in cases where data are naturally unbalanced, such as in road accidents, the classification algorithms tend to ignore the less represented categories, in order to optimize the overall precision of the model. In neglecting these distortions, the ANN algorithm fails to predict and classify the most relevant classes, which are generally less frequent in the database, such as accidents involving victims (fatal and non-fatal). In addition, the size of the database may also influence the results obtained, since the database must be large enough to ensure that the subsets of training and test data are as homogeneous as possible. Conversely, accident databases have a reduced number of observations when compared to databases in the area of transportation for planning and analysis of demand, among others. Recommendations for future work include the study of database balancing methods, as well as testing different information transfer functions between ANN layers. In addition, it is recommended to explore, where possible, databases of different dimensions, since a larger

number of data can increase the generalization power and hence the data adjustment to the ANN model. Thus, regardless of the structural flexibility of the ANN in the selection of variables and the construction of stochastic models, it is suggested investigating other approaches that are based on network structures, such as complex networks that are suitable for studying random phenomena of complex nature, derived from multiple causes.

Acknowledgments

The authors would like to thank the Coordination of Improvement of Higher Education Personnel (CAPES) and the grant received by the National Council for Scientific and Technological Development (CNPq) process 304683/2015-9 for the financial support to the development of this research.

Share and Cite