Diagnosis of Diabetes Mellitus Using Statistical Methods and Machine Learning Algorithms
* Author to whom correspondence should be addressed.
Sigma Journal of Engineering and Natural Sciences 2018, Vol. 36, Issue 4, pp. 1265-1282; doi.org/10.62051/ytu.sigma-journal-of-engineering-and-natural-sciences-diagnosis-of-diabetes-mellitus-using-statistical-methods-and-machine-learning-al
Abstract
Keywords: Classification; diabetes mellitus; machine learning; prediction; statistical methods.
1. Introduction
Diabetes mellitus is a group of metabolic disorders with one common manifestation: elevated blood sugar or hyperglycemia [1]. The detection and the diagnosis of the diabetes is the most crucial point due to chronic hyperglycemia causes damage to the eye, kidney, nerves, heart, and blood vessels which causes the permanent damages. This contingency makes the diagnosis of the diabetes indeed important. The traditional diagnosis methods may be more painful and slower such as blood analysis [2]. A physician commonly determines decisions by evaluating the current blood analysis results of a patient. Therefore, diagnose of diabetes for the physicians and the patient is very difficult matter. For this reason, a lot of the intelligent diagnosis system for diabetes has been evolved by inspiring human-being biological constructers [3]. These evolved methods predict whether the probable patient suffers diabetes mellitus or not without including any surgeon progress [4]. In this context, this study aims to present a comprehensive literature review for the diagnosis of diabetes mellitus. The remainder of this study is organized as follows: the next section provides a comprehensive review of relevant
Corresponding Author: e-mail: pkl.ebru@gmail.com, tel: (362) 312 19 19 / 1054
literature. In the third part, the mostly used techniques are described in detail. In section 4, the performance criteria are described which are the most widely used in the evaluation of classification algorithms in the literature. Chapter 5 gives general and comprehensive information about the studies in the literature. In the final part, the results of these study and literature review are discussed.
2. Literature Review
Many studies can be found for the early diagnosis of diabetes mellitus in the literature. These studies can be summarized as follows: Boyle et al. [5] managed a duality analysis to predict the diabetes population in 2050. Another study evaluated the average cost value caused by diabetes mellitus [6]. They estimated diabetes-related costs by using Cardiff Diabetes Cost-Benefit Model which takes into consideration the probabilistic processes. Some authors constructed a micro simulation model to evaluate the various scenarios for diabetes population [7]. Upadhyay and Patel [8] proposed a fuzzy classifier models to classify the diabetes condition. Their model allowed a splendid classification performance with 98.88% accuracy rate. The regression based forecasting models have been commonly used by now and the regression models are the one of the oldest prediction models. A various statistical models were derived on diagnosis of diabetes as seen Table 1. Additionally, the definition of the most suitable hybrid methods is essential due to acquire the best results. Moreover, the artificial intelligent technique provides to increase diagnostic accuracy and reduces costs and human resources [28]. Temurtas et al. [29] predicted on the same diabetes data by using Levenberg Marquardt learning algorithm and Probabilistic Neural Network (PNN) with 50 neurons for each hidden layer. Their model gives 82.37% accuracy rate. Polat and Güneş [30] presented a new hybrid model which occurs two stages. At first stage, principal component analysis is applied for reducing the number of features. At second stage, they predict diabetic condition. They acquire 89.47% accuracy rate. Doğantekin et al. [31] applied a hybrid method which is integrated Linear Discriminant Analysis (LDA) and ANFIS. Their prediction accuracy is 84.61%. Besides, Kala et al. [32] compared to three different neural network methods which are ANFIS and Evolutionary Artificial Neural Networks (EANN). According to their results, the best result is reach by EANN with 77.38 % accuracy rate. Drezet and Harrison [33] and Georga et al. [34] predicted by using support vector regression integrated with other technique. According to Karahoca et al. [35], the ANFIS provides the better results in comparison to Multinomial Logistic Regression under condition that dependent variables has more than two values such fuzzy numbers. They had a different database and considered glucose rate as input variable. Their error rate (RMSE) is 0.17%. Sharifi et al. [36] argued a hierarchical takagi-sugeno type fuzzy system for diabetes mellitus forecasting. Their accuracy rate is 78.73% which is the best result comparing to some conventional methods. Smith et al. [37] applied a neural network method with ADAP learning algorithm and their sensitivity rate was calculated as 76% while the accuracy rate was not calculated. Former intelligent methods are combined with other algorithms because the developed hybrid method’s efficiency allows reaching the better results than former intelligent methods. Table 1. The classification of the studies based on statistical methods Ref.
Methodology
Independent Variables Age, gender, frequency of diabetes Age, gender, weight, educational status, body mass index, waist circumference, fasting blood sugar. Age, usage of alcohol, usage of cigarette, physical activity, usage contraceptive, chronic pancreas history,
Methodology
Independent Variables hypertension history, education level, monthly income, weight, standing and sitting height lengths, waist and hip circumference. Age, gender, body mass index, normal or overweight status, obesity status, type 2 diabetes status, health insurance. Age, body mass index, diabetes history, social status, ethnicity. Ethnicity, age, poverty status. Age, race, waist circumference, hypertension status, cholesterol status, physical activity, smoking, alcohol use, diabetes in the family. Blood glucose levels, cholesterol, triglyceride, body mass index, blood pressures, age, monthly income. Age, Diabetes predigree, height, systolic blood pressure, hip circumference measure, body mass index, cholesterol, non-HDL blood pressure, triglyceride, fasting blood sugar, physical activity, c-reactive protein, family income, smoking, alcohol use, use of lıquid-decreasing drugs . Age, gender. Age, gender, educational status, income status, the birth, smoking, alcohol use, physical activity, body mass index. Weight, height, waist and hip measures, HbA1c, glucose, uric acid, AST, ALT, GGT. Age, gestational status, body mass index, diabetes status in family tree, blood pressure, fasting sugar, fasting insulin concentration. The number of pregnancies, plasma glucose concentration, diastolic blood pressure, triceps subcutaneous thickness, 2 hour serum insulin, diabetes pedigree function, age. Having a parent with diabetes, having a parent with 2 diabetes, having a sibling with at least 1 diabetes, age, height, waist circumference, hypertension predisposition, physical activity, smoking, whole grain consumption, coffee consumption, red meat consumption. Waist circumference measure, body mass index, smoking, use of hypertension drugs, blood pressure values, plasma glucose ratio, HbA1c, cholesterol values, triglyceride ratio. Chronic liver diseases, high alanine aminotransferase, reflux state, hypertension, hA1c ratio. Age, body mass index, overwight, obesity, hypertension, diabetic state. The number of pregnancies, plasma glucose concentration, diastolic blood pressure, triceps subcutaneous thickness, 2 hour serum insulin, diabetes pedigree function, age.
The classification of major studies that use machine learning algorithms in literature is presented in Table 2.
Table 2. The classification of the studies based on machine learning algorithms Ref. Year
Methodology
Lasso Statistical Analysis with Bayesian Approach Support Vector Machine with Principal Component Analysis Artificial Neural Network
Independent Variables The number of pregnancies, plasma glucose concentration, diastolic blood pressure, triceps subcutaneous thickness, 2 hour serum insulin, diabetes pedigree function, age. Age, diagnosis time, HbA1c, blood sugar, triglyceride, cholesterol, body mass index, systolic blood pressure, diastolic blood pressure. Age, gender, family history of diabetes, body mass index, waist and hip measurements, systolic blood pressure, diastolic blood pressure, cholesterol, fasting blood glucose, 2-hour glucose. The number of pregnancies, plasma glucose concentration, diastolic blood pressure, triceps subcutaneous thickness, 2 hour serum insulin, diabetes pedigree function, age. The number of pregnancies, plasma glucose concentration, diastolic blood pressure, triceps subcutaneous thickness, 2 hour serum insulin, diabetes pedigree function, age. Smoking, usage of alcohol, body mass index, waist measure, family history, blood pressure. A1c1, A1c2, Sys-BP1, Sys-BP2, Dias-BP1, Dias-BP2, SerumGLU1, Serum-GLU2,body mass index, keratin, HDL, MDRD, triglesirid, race, gender, age, diabetes condition. Heart attack history, cholesterol level, length. Age, height, waist circumference, hypertension predisposition, physical activity, smoking, full-grain consumption, coffee consumption, red meat consumption, alcohol consumption. The number of pregnancies, plasma glucose concentration, diastolic blood pressure, triceps subcutaneous thickness, 2 hour serum insulin, diabetes pedigree function, age. The number of pregnancies, plasma glucose concentration, diastolic blood pressure, triceps subcutaneous thickness, 2 hour serum insulin, diabetes pedigree function, age. The number of pregnancies, plasma glucose concentration, diastolic blood pressure, triceps subcutaneous thickness, 2 hour serum insulin, diabetes pedigree function, age. Age, gender, body mass index, blood pressure, blood pressure, plasma glucose ratio, triceps skin fold thickness, 2-hour serum insulin, diabetes pedigree, cholesterol, weight. The number of pregnancies, plasma glucose concentration, diastolic blood pressure, triceps subcutaneous thickness, 2 hour serum insulin, diabetes pedigree function, age. Age, diabetes predegree, weight, gender, usage of alcohol and cigarette, frequency of thirst, urinary frequency, height, feeling of fatigue easily. The number of pregnancies, plasma glucose concentration, diastolic blood pressure, triceps subcutaneous thickness, 2 hour serum insulin, diabetes pedigree function, age. The number of pregnancies, plasma glucose concentration, diastolic blood pressure, triceps subcutaneous thickness, 2 hour serum insulin, diabetes pedigree function, age. Age, occupational status, nutritional status. Hair and urine values(Li, Cr, Fe, Zn, Cu, Mg, Ni,V.) Brain Cancer Indications.
Methodology
Cognitive Development Optimization Algorithm based Support Vector
Blood pressure values, anthropometric measures, fasting blood sugar. Total bilirubin, BUN, Keratinine, Glucose AC, Glucose PC, Thyroxine, Uric Acid, Cholesterol, Triglyceride, HDL, Glucose, Gene, Age, Vital Capacity, Estimated vital capacity, FEV1, PFR, Albumin, Total Protein, SGOT, SGPT, ELDL, LDL. Gender, age, height, weight, body mass index, diabetes history, pregnancy history, gestational diabetes history, abortion history, high blood pressure history, use of blood pressure drugs and history, systolic and diastolic blood pressure. The number of pregnancies, plasma glucose concentration, diastolic blood pressure, triceps subcutaneous thickness, 2 hour serum insulin, diabetes pedigree function, age.
On the other hand, there have been probabilistic approaches to the diagnosis like Markov Models [62, 63]. A classification of markov models based studies in literature on the diagnosis of diabetes mellitus is given in Table 3. Table 3. The classification of the studies based on Markov models Ref. Year
Methodology
Markov Model Markov Model Markov Model Markov Model Discrete Markov Model Monte Carlo with Markov Model
Markov Status Age, race, ethnicity, gender. Undiagnosed diabetes status, diagnosed diabetes status, death status. Gender, ethnicity, blood pressure, cholesterol level, GHb level, duration of diabetes. Diabetes status, Obesity status, Smoking. Diabetes status, Obesity status, Smoking. Diabetes status, Obesity status, Smoking. Diabetes status, Obesity status, Smoking. Demographic changes, disease dynamics, age and gender. Death, fertility, migration, body mass index, genotype, participation in work. Age, ethnicity, marital status, level of education, occupation, family income, relatives status, body mass index, physical activities, smoking, sleep duration, family history of diabetes. Diabetes status, Obesity status, Smoking. Undiagnosis diabetic state, Type 2 Diabetes status, Type 1 Diabetes status, death.
In order to improve the performance measures of machine learning algorithms, hybridization approach with optimization algorithms has been used in recent years [76]. The idea that machine learning algorithms can be hybridized with optimization algorithms is first proposed by Davis [77]. Later on, this work was first conducted by Kelly and Davis [78]. The authors showed that the K-nearest neighbor algorithm was hybridized with the genetic algorithms and increased the performance values. Although the introduction of a new idea by Kelly and Davis in the literature began in the 1990s, the full dissemination of the idea became possible from the 2000s. Some highly cited studies are summarized in Table 4.
Table 4. Hybrid Models Ref. [79] [80] [81] [82] [83] [84] [85] [86] [87] [88] [89] [90] [91] [92] [93] [94] [95] [96] [97] [98] [99] [100] [101] [102] [103] [104]
Year 1990 1996 2001 2003 2004 2005 2006 2006 2006 2007 2007 2007 2007 2007 2008 2008 2009 2009 2010 2010 2010 2010 2012 2013 2013 2014
Method
Machine Learning Algorithms Artifcial Neural Networks Support Vector Machine Artifcial Neural Networks Decision Tree Support Vector Machine Support Vector Machine Support Vector Machine
Nonmem
Support Vector Machine K-Nearest Neighbors K-Nearest Neighbors Artifcial Neural Networks Artifcial Neural Networks Support Vector Machine Support Vector Machine K Harmonic Means Artifcial Neural Networks Support Vector Machine Support Vector Machine K-Nearest Neighbors K-means Algorithm Support Vector Machine Support Vector Machine Artifcial Neural Networks Support Vector Machine
Hybrid Genetic Algorithm Multi-Variable Discriminant Analysis Independent Component Analysis Decision Tree Genetic Algorithm Genetic Algorithm Genetic Algorithm Genetic Algorithm Genetic Algorithm Genetic Algorithm Fuzzy Artificial Immune Recognition System Tabu Search Algorithm Genetic Algorithm Ant Colony Optimization Particle Swarm Optimization Genetic Algorithm Particle Swarm Optimization Decision Tree Genetic Algorithm Independent Component Analysis Genetic Algorithm Particle Swarm Optimization Simulated Annealing Particle Swarm Optimization Genetic Algorithm K-Means Algorithm
3. Methodologies
The machine learning algorithms are frequently used in the Diabetes Mellitus prediction and classification problems as can be figured out from the previous sections [106, 107, 108, 109]. In this section, the most 4 popular machine learning algorithms are introduced and explained in their general form.
3.1. Decision Tree
Decision Tree (DT) is the one of the supervised learning algorithm that is mostly used in classification problems and works on both categorical and continuous input and output variables [110]. It is one of the most widely used and practical methods for inductive inference. Decision trees learn and train themselves from given examples and predict for unseen situations. Each branch node represents a choice between a number of alternatives and each leaf node represents a decision. In DT, there have been some measures that can help us in selecting the best choice such entropy, gained information. In data mining, entropy is a measure of the uncertainty about a source of messages or a degree of disorganization in the data set. Given a collection S containing positive and negative examples of some target concept, the entropy of S relative to this boolean classification is calculated as in Equation (1).
More precisely, the information gain Gain (S, A) of an attribute A relative to a collection of examples S is defined as in Eq(2). 𝑆
S = Each value v of all possible values of attribute A Sv = Subset of S for which attribute A has value v |Sv| = Number of elements in Sv |S| = Number of elements in S Decision trees, while providing easy to view illustrations, can also be unwieldy. Even data that is perfectly divided into classes and uses only simple threshold tests may require a large decision tree. Large trees are not intelligible, and pose presentation difficulties.
3.2. Naive Bayes
Naive Bayes classifier is a useful algorithm for the classification problem and is based on Bayes’ theorem with independence assumptions between predictors [111]. Bayes theorem provides a way of calculating the posterior probability for each class, P(c|x), from P(c), P(x), and P(x|c). Naive Bayes classifier assume that the effect of the value of a predictor (x) on a given class (c) is independent of the values of other predictors. This assumption is called class conditional independence (Eq. 3) 𝑃(𝑐 |𝑥 ) =
P(c|x) is the posterior probability of class (target) given predictor (attribute). P(c) is the prior probability of class. P(x|c) is the likelihood which is the probability of predictor given class. P(x) is the prior probability of predictor. Naïve bayes is easy to implement and fast to solve problems. It scales linearly with the number of predictors and data points and can be used for both binary and multiclass classification problems.
3.3. Support Vector Machine
In machine learning, Support Vector Machines (SVM) are supervised learning models with associated learning algorithms that analyze data used for classification and also regression analysis [112, 113]. SVM is composed in the framework of statistical learning, which has been developed by Vapnik and Chervonenkis. SVR maps the input data x into a higher dimensional feature space through a nonlinear mapping and then a linear regression problem is obtained and solved in this feature space. With the given training data {(x1,y1),…, (xi,yi),…, (xn,yn)}, the mapping function can be formulates as in Eq (4). 𝑓(𝑥) = ∑𝑛𝑖=1 𝑤𝑖 𝑖 𝑥𝑖 + 𝑏
Where ωi and b are the parameters that need to be defined. SVR is to find a function f(x) that has at most deviation from the actually obtained targets yi for all the training data and at the same time is as flat as possible. Flatness in this case means to reduce the model complexity by minimizing ||ω||2, so that this problem can be written as an optimization problem as seen in Eq(5) and Eq (6). 1
Equation (7) defines a constrained optimization problem. Equation (8) shows the solution of this problem. 1
While SVM has a regularization parameter, which makes the user think about avoiding overfitting, it does not give class probabilities and being rather cumbersome for multiclass problems.
3.4. Artificial Neural Network
The purpose of ANNs is to present a development to mimic the basic biological neural systems including the human brain [114]. ANNs have a number of interconnected simple processing points. If an input signal is picked by each node and operated through an activation or transfer function and a transformed output signal is generated. Though each function is implemented by each individual neuron quite slowly, a network can execute an amazing number of tasks efficiently. As an advantage, ANN is the flexibility in changing the encoding of the data to fit different statements of the problem and is capable to conform to the real world.
4. Machine Learning Performance Measures
The confusion matrix is a two by two table that contains four outcomes produced by a binary classifier. Various measures, such as error-rate, accuracy, specificity, sensitivity, and precision, are derived from the confusion matrix (Table 5).
Prediction outcome p N True False Positive Negative False True Positive Negative P N
There are basic performance metrics that can be used to evaluate the methods applied in machine learning. The most popular and basic method used to measure model performance is the accuracy of the model. The accuracy rate is the ratio of the number of true classified samples (TP + TN) to the total number of samples (TP + TN + FP + FN) (Equation 9). 𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 =
Precision is the ratio of the number of True Positive (TP) samples predicted as class 1 to the total number of samples (TP + FP) predicted as class 1 (Equation 10). 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =
Sensitivity is the ratio of the number of correctly classified positive samples (TP) to the total number of positive samples (TP + FN) (Equation 11). 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 =
Precision and sensitivity metrics alone are not enough to make a meaningful comparison in the application comprised from a few machine learning algorithms. It can be obtained better comparison results by evaluating both criteria together. Therefore, F-measure (F) is used for comparing the algorithms. The F-criterion is the harmonic mean of the precision (P) and the sensitivity (S) (Equation 12). 𝐹 − 𝑀𝑒𝑎𝑠𝑢𝑟𝑒 =
5. Descriptive Analysis
In the present, machine learning algorithms are mostly popular among the other techniques due to yield outstanding classification performance according to former techniques. Nevertheless, it can be constructed much more effective machine learning algorithms by hybridizing some optimization techniques. The distribution of articles by year of publication is shown in Figure 1. It is obvious that publications which are related to application of machine learning techniques in diagnosis diabetes mellitus have increased significantly from 2007 to 2017. In 2006, the largest increase has taken place between 2015-2016 years with 75%.
Figure 1. Machine learning in diabetes mellitus Table 6 shows the distribution of articles by countries. Articles related to application of machine learning techniques in diabetes diagnosis are distributed across 10 countries. Of these, “USA”, which focuses on the knowledge of the application of expert and intelligent systems in diabetes more than 25% (110 of 425 articles) of the total number of articles published. Table 7 shows the distribution of science category by classification and prediction on diabetes from Web Of Science. Among 425 papers which have been applied in diabetes diagnosis, artificial intelligence field is the most commonly used in literature. It has been described in 96 (22.5%) out of 425 articles in total. Following are computer science interdisciplinary applications and engineering electrical electronic which have been described in 74 (17.41%) and in 74 (17.41%) fields respectively.
Table 6. Studies by countries Country USA India China England Australia Turkey Canada Japan Malaysia South Korea
Ratio 0.25882 0.14824 0.10353 0.07059 0.04941 0.04471 0.03529 0.03294 0.03294 0.03294
Table 7. Studies by work categories Work Categories Amount Ratio Computer Science Artificial Intelligence 96 0.22588 Computer Science Interdisciplinary Applications 74 0.17412 Engineering Electrical Electronic 74 0.17412 Computer Science Theory Methods 66 0.15529 Medical Informatics 64 0.15059 Engineering Biomedical 55 0.12941 Computer Science Information Systems 50 0.11765 Mathematical Computational Biology 43 0.10118 Health Care Sciences Services 27 0.06353 Endocrinology Metabolism 23 0.05412 Table 8 shows the top 10 of articles by journal. Articles related to application of machine learning techniques in diabetes diagnosis are distributed across 55 journals. Of these, “Expert Systems with Applications”, which focuses on the knowledge of the application of expert and intelligent systems in diabetes diagnosis, contains more than 3% (14 of 425 articles) of the total number of articles published. Table 8. Studies by journals Journals Amount Ratio Expert systems with applications 14 0.03294 Lecture notes in computer science 10 0.02353 Artificial intelligence in medicine 9 0.02118 Plos one 9 0.02118 Journal of biomedical informatics 7 0.01647 Ieee engineering in medicine and biology society conference proceedings 6 0.01412 Journal of medical systems 6 0.01412 Lecture notes in artificial intelligence 6 0.01412 Computer methods and programs in biomedicine 5 0.01176 Diabetes 5 0.01176 The distribution of articles by machine learning techniques is shown in Figure 2. It is obvious that support vector machines were significantly used in application of machine learning
techniques in diagnosis diabetes mellitus compared to other techniques. It can be said that the most 4 popular machine learning algorithm are support vector machines, naïve bayes, neural network and decision consecutively.
6. Discussion And Conclusions
In this study, firstly a detailed classification of studies in literature about the diagnosis of diabetes mellitus. The diagnosis problem of the diabetes mellitus is the oldest research topics. This problem is the crucial problem due to expect that the amount of the diabetes patient may increase until 2025. While some papers perform to predict the spread of DM, some papers perform to analyze the diabetic condition of people. The regression models were used extensively to forecast the diabetic condition of persons in earlier years due to the statistical methods is the former techniques among the forecasting techniques. The Markov models emerged to predict diabetic condition people by time. These models have been widely placed in the literature due to has an advantage which is taking into consideration probabilistic factors. Among the 97 articles, 31 described support vector machine in the diabetes classification problems. Support vector machine can be applied easily in classification due to allow get only the binary outputs. Thus, it is not surprising that support vector machine were used in a wide range of diabetes classification. Naïve bayes and neural networks techniques rank after support vector machine in popularity of application diagnosis of in diabetes. The majority of diabetes studies in the literature have been conducted on the PIMA Indian data set with 0.67% [116]. The performance measures on this dataset show a change from 73.83% to 96.00%. Performance measures are higher in diabetes prediction and classification problems that are conducted in data sets collected from different sources. This study might have some limitations such that only surveyed articles published between 2000 and 2017, which were extracted based on combination of keywords search of “diabetes” and “classification” “prediction “or “machine learning”. In the literature, traditional machine learning algorithms have been replaced by models that hybridize with optimization algorithms over time. As the optimization algorithm, it was seemed that the most commonly used and best-resultant method is genetic algorithm. In addition, nearly half of the hybrid models proposed in the literature are hybridized with genetic algorithms (13 of 27). Following is particle swarm optimization algorithm with 14.81% (4 of 27).
It is obvious that the combination of different machine learning algorithms and optimization models can lead to more meaningful and powerful results. Among machine learning algorithms which are employed in hybridization, neural networks and support vector machines are widely used in diagnosis of diabetes. It might be obtained more powerful results by hybridizing new prediction and classification methods such as Extreme Learning Machines. On the other hand, to be used genetic algorithm in hybridization yields good results and increases the performance of traditional machine learning algorithms in general.
Share and Cite
PEKEL, E.; ÖZCAN, T. Diagnosis of Diabetes Mellitus Using Statistical Methods and Machine Learning Algorithms. Sigma Journal of Engineering and Natural Sciences 2018, Vol. 36, pp. 1265-1282. https://doi.org/10.62051/ytu.sigma-journal-of-engineering-and-natural-sciences-diagnosis-of-diabetes-mellitus-using-statistical-methods-and-machine-learning-al