Machine learning classification models for the patients who have heart failure

Abstract

Keywords: Heart Failure; Machine Learning; Classification Algorithms; Principal Component Analysis; CatBoost

Introduction

Heart failure (HF) is a cardiovascular disease that causes substantial morbidity and mortality in a growing number of people around the World [1]. Heart failure affects about

6.5 million people in the United States, more than 14 million people in Europe, and 26 million people worldwide, and the number continues to rise [1]. Heart failure affects more men than women, and its incidence rises rapidly as people get older [2].

*Corresponding author. *E-mail address: makar@yildiz.edu.tr This paper was recommended for publication in revised form by Regional Editor Ahmet Selim Dalkilic Published by Yıldız Technical University Press, İstanbul, Turkey Copyright 2021, Yıldız Technical University. This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Heart failure is a cardiac structural or functional disorder that causes the heart to fail to provide enough oxygen to meet the metabolic needs of the tissues despite normal filling pressures or only at the expense of increased filling pressures. Heart failure is a clinical syndrome caused by structural or functional impairment in the heart, in which patients have typical symptoms such as shortness of breath, ankle swelling and weakness and signs such as elevated jugular venous pressure, pulmonary crackles, and displaced apex beat [3]. Heart failure (HF) is generally classified based on the left ventricular ejection fraction (LVEF), with HF with LVEF greater than 50% being referred to as HF with preserved ejection fraction (HFpEF) and HF with LVEF less than 50% being referred to as HF with reduced ejection fraction (HFrEF). In recent years, the HFrEF has been split into two parts. The LVEF value is classified as a mid-range ejection fraction in the 40%-49% range and a reduced ejection fraction when the LVEF value is lower than 40% [4]. Left ventricular systolic dysfunction is defined as an LVEF less than 40%. In this work, we use various machine learning strategies to produce the best possible results according to various performance metrics. In the related work section, the studies conducted with this data set are mentioned. In the Data Set section, the variables in the data set were mentioned and examined. The methods applied and the results obtained are mentioned in the Methods section. In the Conclusion section, it is mentioned what purpose this study can be used for.

Related WORK

The data set used in this paper was made available in an paper published in 2017. In that paper [5], the survival status of 299 patients who have heart failure in Pakistan in 2015 was analyzed by Ahmad, Munir, Bhatti, Aftab, and Raza using the Cox Regression and Kaplan Meier plot. Following that, a gender-based survival analysis study was conducted by Zahid, Ramzan, Faisal, and Hussain [6] on this dataset using statistical techniques such as Cox Regression in 2019. Finally, Chicco and Jurman [7] applied machine learning algorithms to this data set to determine the patients’ survival status in 2020. Chicco and Jurman [7] stated that the survival analysis of patients can be performed by looking at the creatinine and ejection fraction features in the data set using machine learning algorithms. By adding the time feature to these two features, logistic regression is the method that gave the most successful result among various machine learning algorithms [7].

Data SET

Our data set consists of 299 patients who went to the Faisalabad-Pakistan Cardiology Institute and Allied

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 235−244, February, 2024

hospital between April-December (2015). The dataset includes 105 female patients and 194 male patients. In the data set, the age of the patients, whether they have a smoking habit, various analysis results and whether they have diabetes, blood pressure, and anaemia are given. The paper [5] did not mention whether patients had another significant disease. The ages of the patients are between 40 and 95 and the average age is 60.83. Creatinine phosphokinase (CPK), platelets, ejection fraction, creatinine, sodium values were examined in the blood of patients. These variables are given as continuous variables in the dataset. Smoking, diabetes, anemia, blood pressure (BP) and gender are categorical variables in the dataset. In addition to these variables, the patients’ death or survival times are given in days, a variable known as time. All of the patients have left ventricular systolic dysfunction and are classified as NYHA class III and IV. Time is 4-285 days with an average of 130 days [5]. The data set contains no missing values. Using these variables, we aim to classify patients’ chances of death and survival as accurately as possible. In the data set, death=1 denotes a deceased patient, while death=0 denotes a survived patient. The enzyme CPK catalyzes the reaction of creatine and adenosine triphosphate (ATP) into phosphocreatine and adenosine diphosphate (ADP). CPK levels of 20 to 200 IU/L are considered natural. CPK levels may be affected by a variety of factors, involving rhabdomyolysis, heart disease, kidney disease, and even some drugs [8]. Platelets are specialized disk-shaped cells in the bloodstream that help to build blood clots and are involved in heart attacks, strokes, and peripheral vascular disease [9]. In patients with HFrEF, there is a connection between platelets and increased mortality. Platelets may be a way to predict poor results in patients with HFrEF [10]. Creatinine refers to serum creatinine, which is a variable included in the data set. Higher creatinine levels and a greater rise in serum creatinine have been linked to a longer stay in the hospital, higher long term mortality, and higher rehospitalization rates [11]. Ejection fraction is the percentage of blood volume ejected in each cardiac cycle. Ejection fraction shows left ventricular systolic performance [12]. Sodium is a mineral that is used in diets all over the world. Sodium is essential for maintaining appropriate blood volume and blood pressure [13]. The total number of patients who died was 96, while the number of patients who survived was 203, as shown in Table 1. Therefore, we may assume that the data set is unbalanced in this situation. The means of continuous variables by dead, survived, and total patients are shown in Table 2.

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 235−244, February, 2024

Table 1. Distribution of patients by categorical variables Categorical Variables

Table 2. Means of continuous variables by patients Continuous Variables

Materials And Methods

It is possible for the model to obtain more successful results by determining the various parameter values in each classification method with the GridSearch CV [14] method. In addition, more successful results can be obtained with various changes on the kernels in the support vector machine method. For example, Akar and Sirakov [15] applied classification methods on the skin lesions dataset by increasing the size of the data using Clifford algebra. Apart from increasing the size by using the kernel, Akar, Sirakov, and Mete [16] obtained a new kernel using Clifford algebra and obtained more successful results with these kernels. Principal Component Analysis (PCA) Principal Component Analysis (PCA) can be used to simplify practically any data matrix. A model of the behavior of a physical or chemical system can be created using PCA in conjunction with a well chosen set of objects and variables [17]. Logistic Regression Many of the fundamental presumptions of ordinary least squares-based linear regression models, such as the linearity of the relationship between the dependent and independent variables, the normality of the error distribution,

and the measurement level of the independent variables, are not true in the case of logistic regression. Because it uses a non-linear log transformation of the linear regression, logistic regression can deal with relationships between dependent and independent variables that are not linear [18]. Naïve Bayes A given example that is described by its feature vector is given the most likely class using a Bayesian classifier. By assuming that features are independent of class, that is, where is a feature vector and is a class, learning such classifiers can be considerably sped up. In spite of this irrational presumption, the resulting classifier, naïve Bayes, is very effective in practice, frequently outperforming much more sophisticated methods. Numerous real-world applications, such as text classification, medical diagnosis, and system performance monitoring, have demonstrated the efficacy of naïve Bayes [19]. K-Nearest Neighbors (KNN) The k-Nearest Neighbor (KNN) approach applies the classification of the closest of a group of previously classified points to an unclassified sample point [20].

Support Vector Machine (SVM) Support Vector Machine (SVM) maximizes the geometric margin while minimizing the empirical classification error. For a specific kernel, selecting the right kernel function and parameter values is crucial for the amount of data available. Text categorization, handwritten digit recognition, tone recognition, image classification and object detection, micro-array gene expression data analysis, and data classification are just a few of the real-world issues that SVM have been used to solve [21]. Artificial Neural Network (ANN) An Artificial Neural Network (ANN) is a simulation of the human brain. The ANN is made up of processing units called neurons. An artificial neuron tries to copy the structure and behavior of a natural neuron. A neuron is made up of inputs and outputs. The output can be thought of as a synapse on the axon in the human brain, the input as dendrites in the human brain. Neurons have the function of determining the activation of neurons. Data classification, pattern recognition and application of data that are not clear can be carried out using ANN [22]. Decision Tree By grouping the original inputs with major subgroups, the decision tree simplifies complex relationships among input variables and target variables. A method of classifying the population into branches such as segments that are an inverted tree with root nodes, internal nodes and leaf nodes is used. The decision tree algorithm is non-parametric, which allows it to effectively handle large, complex datasets and does not require a complicated parametric structure. In particular when used with a small data set the main disadvantage is that the Decision Tree algorithm is prone to overfitting and underfitting [23]. Random Forest Multiple decision trees are generated by forest; randomization occurs in two different ways: random data sampling for bootstrap samples, as is done in bagging, and random feature selection for creating individual base decision trees. The main factors that determine a random forest classifier’s generalization error are the power of each decision tree classifier and the correlation among base trees. An efficient method for estimating missing data, random forest has the ability to process thousands of input variables without deleting any, provides estimates of significant variables, generates an internal, unbiased generalization error estimate as the forest grows, and maintains accuracy even when a large amount of the data is missing. Random forest classification has been applied in certain areas such as handwriting recognition, detection of hidden web search interfaces, land map classification, multilabel classification [24].

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 235−244, February, 2024

Gradient Boosting A gradient descent based formulation for boosting methods has been developed to establish an association with the statistical framework. It was referred to as a gradient boosting machine because of this formulation of the boost methods and corresponding models. In gradient boosting machines, the learning process is applied consecutively to new models for more accurate estimation of response variables. The basic idea of the gradient boosting algorithm is to build new baselearners that are maximally correlated with the negative gradient of the loss function associated with the entire ensemble [25]. XGBoost A popular and highly efficient machine learning technique is tree boosting. XGBoost is a scalable end- to-end tree boosting system. The scalability of XGBoost in all scenarios is the main factor that contributes to its success. In distributed or memory-constrained environments, the system expands to billions of samples and performs more than ten times quicker than currently used popular solutions on a single machine [26]. CatBoost Yandex created CatBoost, an improved version of the gradient enhanced decision trees (GDBT) algorithm. In a variety of classification and regression tasks, it excelled. CatBoost is better at handling categorical features than more sophisticated gradient boosting algorithms like XGBoost and lightBGM. It is advised to use ordered boosting rather than the traditional GDBT gradient estimation approach to handle the gradient bias and prediction drift issues in CatBoost. CatBoost has reduced the need for super parameter tuning [27]. The data set consists of a total of 12 independent variables along with the follow up time variable and a dependent variable, i.e. the death variable. This study aims to create a model that will make the most successful classification according to these variables. Firstly, the “event” expression is replaced with “death” in the data set. If death = 1, the patient is dead, if death = 0, the patient is survived. After that, standard scaler is used to scale the data set by eliminating the categorical variables. The categorical variables are is added later back into the data set. After scaling the continuous variables in the data set, they display a distribution in a certain range as shown in Figure 1. The aim of using standard scaler to scale variables is to see the importance of the variables more clearly by putting the variables in the data set into a certain range and making the model more successful. Pearson correlation coefficient (PCC) [28] is used to examine the relationship between the dependent variable “death” and the independent variables after scaling the data set.

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 235−244, February, 2024

Table 3. Pearson correlation coefficient of variables according to “death” Variable

Figure 1. Distribution of continuous variables after standard scaler.

According to Table 3, the most significant variables for the variable death are age, time, creatinine, and ejection fraction, as shown by the PCC. The methods are applied without removing any variables from the data set. First of all, 33% of the data set is separated as test set and 67% of the data set is separated as train set. In other words, 200 patients are allocated for train set and 99 patients for test set. The random state is used to prevent different results every time we run the model. After these stages, two different methods are applied to the data set. In the first version, 12 independent variables is reduced to 6 components by applying PCA [17] to the model. After that, logistic regression [18], naïve Bayes [19], KNN [20],

SVM [21], ANN [22], decision tree [23], random forest [24], gradient boosting [25], XGBoost [26] and CatBoost [27] classifier algorithms are applied to the model. After applying PCA to the model, the importance level of the 6 components as a percentage according to the random forest feature selection is as in Figure 2. In the second version, the classifier algorithms such as logistic regression, naïve Bayes, KNN, SVM, ANN, decision tree, random forest, gradient boosting, XGBoost, and CatBoost are used. Figure 3 shows the importance level of all variables in percentages using the random forest feature selection.

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 235−244, February, 2024

Results And Discussion

Hyper-parameter optimization has been applied to the classification algorithms in two versions. The most suitable parameters for the model are chosen using the GridSearch CV, and then the performance of the model is measured. Accuracy, MCC [29], sensitivity, specifity, F1 score, ROCAUC, and PR-AUC values are calculated to measure success. Logistic regression is applied to the model with solver ‘liblinear’. Linear and rbf are selected as kernels in SVM and compared two different results. GridSearch CV is used to find the best parameters for all models except naïve Bayes and logistic regression. Table 4 shows the performance measurements of the methods after applying PCA to the model. After applying GridSearch CV optimization, n_neighbors= 3 for KNN, C=0.1 for SVM Linear, C=50 and gamma=0.01 for SVM RBF, activation=’relu’, alpha=0.01, hidden_layer_sizes=(30,30)

and solver=’adam’ for ANN, max_depth=6, min_samples_ split=5 for Decision Tree, max_depth=2, max_features=5, n_estimators=1000, min_samples_split=5 for random forest, learning_rate=0.01, n_estimators=500, max_depth=5, min_samples_split=5 for Gradient Boosting, learning_ rate=0.01, n_estimators=100, max_depth=3, min_samples_ split=2, subsample=0.6 for XGBoost, learning_rate=0.1, iterations=200, depth=5 were selected for CatBoost. SVM Linear is the most successful method in accuracy, MCC, ROC-AUC, PR-AUC metrics. ANN gave the most succesful results in the sensitivity, F1 score. Logistic regression, KNN and SVM Linear in the specifity metric gave the same result, as shown in Table 4. MCC values for all models are shown in Figure 4. SVM Linear has the highest MCC value. In the second version, the methods are applied on 12 independent variables without applying PCA to the model. In this version, no hyper-parameter optimization

Table 4. Methods and performance measurement metrics after applying PCA to the model

SVM RBF

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 235−244, February, 2024

has been made for XGBoost. The lack of hyper-parameter optimization is due to the fact that XGBoost achieves better results without using hyper-parameter tuning. After applying GridSearch CV optimization, n_neighbors=7 for KNN, C=6.0 for SVM Linear, C=50 and gamma=0.01 for SVM RBF, activation=’relu’, alpha=0.005, hidden_layer_ sizes=(50,) and solver=’adam’ for ANN, max_depth=4, min_samples_split=16 for decision tree, max_depth=10, max_features=2, n_estimators=1000, min_samples_ split=10 for random forest, learning_rate=0.01, n_estimators=500, max_depth=3, min_samples_split=10 for gradient boosting, learning_rate=0.01, learning_rate=0.1, iterations=200, depth=5 were chosen for CatBoost. Table 5 shows the results of the performance measurement for the second version, and the CatBoost classification method

yield the most successful results in accuracy, MCC, F1 score, and PR-AUC metrics. On the other hand, the most successful sensitivity result is Gradient Boosting, the most successful specifity result is KNN and the most successful ROC-AUC result is XGBoost. Figure 5 shows the MCC values of the model. Accordingly, it is seen that the highest MCC value belongs to CatBoost, followed by XGBoost and Gradient Boosting classification methods. The version with PCA applied to the model produces noticeably lower successful results in all performance assessment measures when we compare the two versions. In the second version, tree-based machine learning methods such as random forest, Gradient Boosting, XGBoost, and CatBoost produce better results than other methods. Among these methods, CatBoost is the most

Table 5. Methods and performance measurement metrics without PCA

SVM RBF

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 235−244, February, 2024

Table 6. Comparison of the most successful results of the two articles

Pr-Auc

successful method for most of the performance measurement metrics. Table 6 shows a comparison of the most effective machine learning approach for this data set used in Chicco and Jurman’s study [7] with the method used in this study. Logistic Regression produced the best results in Chicco and Jurman’s study. They used logistic regression to analyze ejection fraction, serum creatinine, and time variables, and they averaged the results after running it 100 times. They scaled the data set and split the 80% train set into the 20% test set when using this method. In this study the data set is standardized, and 67% of the training set is split into 33% of the test set. We divide the data set in this manner because the data set is very small, and we want to avoid an unbalanced distribution of the data set as a result. In this work, machine learning algorithms were used on all variables, as opposed to Chicco and Jurman’s study. A comparison of the two results reveals that the CatBoost method produces better results for each performance metric. For instance, while the accuracy in Chicco and Jurman’s study was 0.8380, it is now 0.8788 in this study, and the MCC value was 0.6160 in Chicco and Jurman’s study, but it is now 0.7348 in this paper. Thus, in contrast to the Chicco and Jurman’s study, performance measurement metrics clearly show that our

process of separating the data set, selecting variables, and using the CatBoost machine learning algorithm increases success.

Conclusion

Heart failure (HF) is a serious public health problem that affects about 26 million individuals throughout the world. When compared to people with other chronic conditions and the general population, people with heart failure have a much worse quality of life [30]. Despite advances in HF treatment, the rate of morbidity and mortality is still very high. Mortality rates following HF hospitalization are reported to be 10% after 30 days and 22% after a year [31]. HF patients and the death rate from HF are increasing day by day. In recent years, it has become important to recognize diseases with high mortality rates around the world, such as infectious diseases and cancer [32]. Machine learning can predict the mortality status of patients with HF. This study aims to help determine the patient’s mortality status by looking at various variables. The patient’s condition can be changed by choosing the appropriate treatment according to the mortality situation.

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 235−244, February, 2024

Research Data

The dataset used in this study can be found at the following site. https://plos.figshare.com/articles/dataset/Survival_analysis_of_heart_failure_patients_A_case_study/5227684/1

Acknowledgements

This work was supported by the Research Fund of the Yildiz Technical University, Project Number: FYL-20214416. The authors would like to thank for this support.

Data Availability Statement

The authors confirm that the data that supports the findings of this study are available within the article. Raw data that support the finding of this study are available from the corresponding author, upon reasonable request.

Conflict Of Interest

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics

There are no ethical issues with the publication of this manuscript.

Share and Cite