Cart KNN A hybrid distance-based pre-pruned classification and regression method for the early diagn

Abstract

Keywords: Alzheimer's Disease Prediction; CART-KNN; Confusion Matrix; Decision Tree; K- Nearest Neighbors; Pruning

Introduction

The most prevalent type of dementia that needs intensive medical attention is Alzheimer’s disease (AD). An accurate and timely study of AD prognosis is required to start clinical progress and provide effective patient therapy [1]. AD is a long-term, degenerative brain disease that gradually destroys brain tissue, and impairs thinking and memory [2]. The amount of computing power used by healthcare departments is always growing, and patient data is increased electronically. Although many electronic health records (EHRs) are now easier to access, 80% of the data remains unstructured. Because of this, handling unstructured data with database management software and other conventional techniques is difficult. Some methodologies adopted for EHRs through machine learning and data mining technologies improve the standard and efficiency of healthcare and prescribe proper medicine [3]. Finding undiscovered and meaningful patterns from a large number of pre-existing datasets is a process known as data mining or knowledge discovery. These patterns aid in the comprehension of the historical dataset, the classification of a new set of data, and the creation of data summaries. Data mining categorizes or groups records according to their similarities or differences, hence aiding in identifying deeper patterns within the data[4]. Data mining has been widely applied in various fields over the past few decades, including marketing, retail, finance, stock market forecasting, and medical and healthcare [5]. Machine learning and data mining techniques have been widely employed in medical and healthcare research [6]. These algorithms may be used to categorize distinct subjects based on the similarities in their qualities. The primary goal of this research project is to identify the various phases of Alzheimer’s disease (AD) using machine learning and data mining techniques on the Open Access Series of Imaging Studies (OASIS). Finding the most distinctive feature for each of the several phases of AD within the OASIS dataset is also a sub-objective of this research project. Several studies focus on using machine learning to identify blood-based non-amyloid biomarkers for early Alzheimer’s disease detection, achieving high sensitivity and specificity levels with novel protein panels [7]. Machine learning, particularly the Random Forest model, achieves 90.6% accuracy in early Alzheimer’s detection using neuroimaging biomarkers, emphasizing its robustness and diagnostic advancements in Alzheimer’s disease prediction [8]. Random Forest, XGBoost, and Light GBM were used to predict Alzheimer’s disease risk in individuals aged 65+ using Finnish register data, showing low predictive accuracies and suggesting additional data sources were needed [9]. Gradient boosting, random forest, and support vector machines utilizing EEG signals and genetic information effectively classifies Alzheimer’s disease, achieving high accuracy and offering a comprehensive diagnostic approach beyond imaging limitations [10]. Novel drug discovery for Alzheimer’s disease utilized machine learning to identify compounds targeting key proteins like MTOR and BCL2, offering a multi-targeted

therapeutic approach for AD treatment [11]. An explainable machine learning approach for Alzheimer’s disease classification achieved high accuracy using SVM models and highlighted key factors like MEMORY and JUDGMENT in AD development [12]. Machine learning-based speech analysis can effectively distinguish between different cognitive impairments in the Alzheimer’s disease spectrum and predict cognitive domain performance, offering the potential for early detection and monitoring [13]. In recent research data modeling is quite challenging and complex during big data and short-term prediction. A fusion of CART and KNN models has been proposed to solve this issue. A nonlinear CART model is effective for data porting and mining purposes [14]. KNN, a popular nonparametric regression model for searching similar data in a particular region [15]. A novel hybrid approach of CART and KNN has been experimented with for software fault prediction. The proposed approach achieved an average accuracy of 0.90274 which is much higher than the other nine machine-learning models [16]. This work’s main contributions are: to consider the input feature and perform the autocorrelation for finding out the data pattern similarity by CART model. Secondly, combined a regression tree with weighted k-nearest neighbors and distance-based prepruned classification to create a hybrid technique. The present research is arranged into the following stages: in the literature review section, Research gaps and earlier studies are described. The proposed methodology including dataset description, preprocessing, pruning, and CART-KNN method explained. The final section experimentation and performance assessment of our suggested methodology are covered.

Literature Review

Feature selection and extraction techniques are used in machine learning (ML) algorithms to predict Alzheimer’s disease. The Oasis dataset is used as the basis for the classification process. A quick summary of the various methods used in brain image analysis for brain disease diagnosis is given [17]. Based on the findings, this article discusses several important brain disease diagnoses. This work intends to integrate contemporary research on Alzheimer’s disease using different AI platforms. The most accurate diagnosis approach may be identified by the authors by utilizing 21 brain illness reviews. Deep convolutional autoencoders investigate AD data analysis where the proposed method extracts MRI characteristics that reflect neurodegeneration and cognitive symptoms from MRI images [18].With imaging-derived indicators, MMSE or ADAS11 scores can be utilized with over 80% accuracy to predict the diagnosis of AD. To achieve binary classification, a deep neural network with linked layers is employed [19, 20] and to activate each concealed layer distinct activation function was used. A lack of education, high blood pressure, obesity, hearing loss, depression, diabetes, inactivity, smoking, and social isolation are some of the variables that might increase one’s risk.

Sigma J Eng Nat Sci, Vol. 43, No. 5, pp. 1484−1494, October, 2025

Deep Learning with CNN methods has been proposed to extract MRI features for early Alzheimer’s diagnosis, achieving over 97% accuracy, and enhancing detection in its earliest stages [21]. ML models like Gaussian NB, Decision Tree, and Voting Classifier can predict AD disease early with high accuracy, aiding in successful treatment and minimizing harm. Machine learning algorithms can accurately predict Alzheimer’s disease with a 96% validation accuracy[22]. A CNN-SVM model was investigated for early Alzheimer’s disease prediction using MRI images, achieving 94.44% accuracy on Mild Cognitive Impairment (MCI) subjects [23]. A novel ensemble machine learning approach, incorporating feature selection and adaptive voting, was examined in early Alzheimer’s disease detection. The proposed model achieved 93.92% accuracy outperforming traditional methods by 3.39% [24]. A classification model was proposed using SVM, RF, and FNN algorithms for early detection of Alzheimer’s disease, enhancing efficiency and reducing processing time in medical data analysis [25]. A speech-based machine learning has been designed for early detection of Alzheimer’s disease with a 75.59% accuracy, showing promise for non-invasive diagnosis [26]. Machine learning models like DT, RF, and SVM have been investigated to enable early Alzheimer’s disease prediction with up to 80% accuracy using OASIS data [27]. Machine learning models, like Gaussian Naive Bayes, have been proposed to accurately predict Alzheimer’s disease early by analyzing neuroimaging and cognitive data with a classification accuracy of 96.92% [28]. Machine learning models like DT, RF, SVM, Voting classifiers, and GB have been investigated for the Prediction of early stages of Alzheimer’s disease, achieving 83% maximum accuracy on test data [29]. A novel algorithm has been developed using a modified capsule network on the OASIS dataset, achieving 92.39% accuracy in predicting dementia, and outperforming traditional methods and deep learning classifiers [30]. After reviewing earlier research, we created a hybrid strategy based on decision trees and k-nearest neighbors. The following is a list of the proposed work’s primary contributions: 1. a novel method known as distance-based pruning is suggested to prune decision tree nodes. The last part provides a detailed explanation of the distance-based pruning approach’s processes.

2. Our suggested method reduces the O(n) running cost

of the standard k closest neighbor technique to O (log n) + c explained in section 3. KNNs are introduced to the decision tree’s leaf nodes to lower running costs during the training phase.

4. To improve the effectiveness of conventional k nearest

neighbors, the idea of weights is added in the prediction phase, based on the sigmoid function. The closest neighbors’ distance from the site under consideration inversely determines the weights.

Working Approach

This paper suggests a hybrid method based on weighted k-nearest neighbors, regression trees, and distance-based prepruned categorization. The hybrid CART-KNN approach aims to improve prediction accuracy and generalization by combining the decision tree’s structured, rule-based model with KNN’s instance-based, flexible learning. There are various ways to hybridize CART and KNN, but a common approach involves using CART to preprocess data or make initial predictions and then applying KNN to refine these predictions. The overall process can be performed in three stages: tree-based segmented preprocessing, KNN-based post-processing, and finally prediction of the hybrid model. During the Tree-Based Segmented preprocessing stage CART is used to partition the feature space into distinct regions based on the decision tree’s splits. Now KNN algorithm will only consider neighbors within the same region, potentially leading to more localized and accurate predictions. In KNN as a Post-Processing Step after that KNN refines and boosts lower tree node confidence of CART’s prediction. During the Hybridization of the Predicted Model, the prediction can be done by either combining predictions by using weighted average prediction obtained by CART and KNN or a voting mechanism where the best prediction is considered by a majority vote from both models. This methodology helps Hybrid CART-KNN to reduce the Overfitting of the distribution of data, improve Locality Sensitivity by handling varying data densities, and flexibility for adapting to different types of data and problems. The run time of the proposed algorithm is reduced to O (log n) +c from O(n), where each factor has its significance like the notion of O(n) is linear time, while O (log n) running times divide-and-conquer application because of ideally cutting the work in half every time. m-dimensional points represent independent characteristics in the dataset. A K − Matrix is generated with the following condition; the element eij = 0 if the jth training sample is far from the ith training sample otherwise eij = 1. Following the KNN-matrix computation, the Euclidean distance formula is used to determine the maximum distance Max distance between all points, and a constant parameter called tolerance is added to regulate the creation of decision trees. Each node of DT fulfills the criteria Maxdistance ∗ tolerance, then the KNN matrix stores the leaf node value rather than class labels. The Weighted K-Tree (WK Tree) creation technique is described in full in this portion of the study publication [31]. These are the steps that explain how to construct a WK-Tree: Pseudo Code Let the Input contain, Training Samples X, and Testing Classes Y, and the Output contain Confusion Matrix Step 1: m dimensional space is considered for all training samples and Euclidean distance is measured by eqn. (1). Step 2: KNNs construct a matrix by considering all training samples using eqn. (2). Step 3: Applied CART, a distance-based pre-pruning

Sigma J Eng Nat Sci, Vol. 43, No. 5, pp. 1484−1494, October, 2025

Step 4: KNN stores all the leaf nodes without repetition and removes all duplicate data. Step 5: The decision Tree helps the testing samples to try to reach the leaf node. Step 6: The weighted label of each test point is assigned. Step 7: Based on predicted and actual values Confusion matrix is designed. Step 8: Accuracy, sensitivity, Precision, and F1-Score are calculated with the help of Confusion Matrix Calculation of the Largest Distance The approach’s initial step involves treating all training samples lying in m-dimensional space and computing the maximum distance between them all. Distance in m dimensions was calculated between all training samples [32]. Equation (1) may be used to determine the Euclidean distance or space, (1)

Iterating over the number of dimensions is the variable k while iterating over the number of training samples is the variable i & j. KNN Matrix Generation For every training sample, a n ∗ n matrix of KNN is constructed. The function + c is chosen to determine the closest neighbors. Equation (2) displays an example of a 5 ∗

1. Position j is regarded as the closest neighbor of position

i if the element eij of the K- -matrix is 1, but not the nearest neighbor point if the element eij of the KNN matrix is 0 (Fig. 1).

Decision Tree Creation Using distance-based pre-pruning and the Gini index node splitting approach, a decision tree is constructed [33]. Pre- and post-pruning are the two types of DT node pruning techniques. Since pre-pruning based on each leaf node’s maximum depth from the tree’s root is ineffective, we have instead used pre-pruning based on distance. Distancebased pruning involves multiplying the maximum distance by the constant parameter Tolerance, which is initially set between 0 and 1. Labeling of Testing Samples During the labeling step, the testing sample’s final label is assigned by applying the weighted k-nearest neighbor technique after traversing the classification and regression trees to reach the leaf node [34]. Using equation (3) weights of each neighbor were calculated. (3) Where Li indicates fault and non-fault-prone classes. Distance-Based Pruning The greatest distance between any two points is determined first in distance-based pre-pruning of decision trees, and the decision tree is subsequently pruned using the percentage of the maximum distance. To encompass all locations, the maximum distance is computed [35]. Max distance and the global parameter are computed. Max distance ∗ Tolerance is checked for all nodes, and tolerance is manually set between 1 and 0 according to the point density. Expression of Parameter Max distance is.

Time Complexity Since training time is a one-time expense, our suggested approach’s testing time complexity will be the only thing we talk about [36]. During the prediction phase, the Decision

Sigma J Eng Nat Sci, Vol. 43, No. 5, pp. 1484−1494, October, 2025

Tree traverses in O (log n) time complexity from the root node of each leaf. In our suggested technique, k-nearest neighbors are kept at leaf nodes instead of labels, adding a little constant c to the decision tree’s real testing time complexity. Our suggested method offered total run time complexity is O (log n + c). Data Preprocessing There are missing values and redundant data in the raw dataset [37-40]. Features with missing values are extracted and modified as part of data management. In this study, min-max feature scaling methods proposed to scale the feature between the range (0,1) with the help of minimum value & difference of the range and information gain filtering methods used for feature selection carried out in preparation for the KNN algorithm’s preprocessing of the dataset. Data Analysis Examining the dataset, followed by cleaning, converting, and modeling it into a format that works, is data analysis. Finding useful information that will subsequently be utilized to enhance decision-making is the goal of data analysis [41,42]. The study might be useful, for instance, in assessing the properties of the data and the relationship between co-relation features. Dataset This study uses a dataset from the Open Access Series of Imaging Studies (OASIS) [27] to predict Alzheimer’s disease. Depending upon cross-sectional MRI and longitudinal MRI the data samples are assigned as dementia and

Table 1. List of twelve attributes Identification, Age, Years of Education, Gender, Dominant Hand, Socio-Economic Status (SES), Clinical Dementia Rating, Normalize Whole Brain Volume, Mini-mental state examination, Atlas Scaling Factor, Estimated Total Intracranial Volume, Delay.

non-dementia at a certain time or baseline is the first step. 150 people, ages 60 to 96, whose MRI data are included in the study. The dataset contains 72 non-demented patients and 69 were dementia. This distinction did not change during the study. The twelve OASIS longitudinal dataset characteristics are displayed in Table 1. This includes handling missing values, extracting and transforming features, and so on. Nine rows in the SES column have missing values, which may be fixed by removing the corresponding rows [43,44]. The measurement as a whole is now 141.

Results And Discussion

This article’s suggested method was created and tested on a computer with an 8GB RAM corei5 CPU. This method is developed using Anaconda 3, and its performance is evaluated against various machine learning models [45,46]. Before we proceed to the comparative study of proposed methods with the other three machine learning algorithms we find the effectiveness of the CART-KNN with the other two fundamental machine learning algorithms KNN and DT in Table 2. Cohen Kappa ranking is used to determine the degree of agreement between two independent raters and AUC (Area under the curve) for binary classification of the dataset are the main metrics to evaluate. Accuracy This study employs a suggested methodology to https:// www.kaggle.com/jboysen/mri-andalzheimers?select=oasis_cross-sectional.csv is where you can find this data. Table 3 presents an accuracy performance metric comparison between our suggested method and three SVM, RF, CARTKNN, and decision tree variants (CART). To complete the experiment, set the tolerance value to 0.05 [47]. The accuracy of our suggested method is compared to various machine-learning techniques in Figure 2. The accuracy distribution of our suggested method demonstrates unequivocally that it performs better in this article than alternative machine learning techniques [48].

Table 2. Comparison study of the proposed approach and its fundamental approaches

Cart-Knn

Table 3. A comparative study based on accuracy for constant tolerance =0.05 s

Cart-Knn

Sigma J Eng Nat Sci, Vol. 43, No. 5, pp. 1484−1494, October, 2025

Figure 2. Comparison of accuracy in terms of bar plots. Other Performance Metrics The average accuracy, F1-score, and recall, for all methods compared on datasets related to Alzheimer’s disease are displayed in Table 4. This article’s suggested strategy outperforms all other approaches utilized in the comparison in terms of accuracy and f1-score performance criteria. A graphical depiction of the accuracy, recall, and F1-score comparison research is shown in Figure 3. Run Time The run time of the proposed model is O (log n) + c, which is compared with another traditional model in Figure 4.

Table 5 presents a comparison with previous research on the OASIS dataset for AD prediction using several machine-learning algorithms. Kavitha and others. [29] demonstrated 83% accuracy in AD prediction using ML on the OASIS dataset, Basheer et al. Using a combination of deep learning and machine learning, [30] achieved a model accuracy of around 92.39%. Six distinct ML approaches were developed by Uddin et al. [22] and the developers of one of them achieved a maximum accuracy of 96% for the voting algorithm. Comparison of Prediction Accuracy of the Model In this part, we assessed the two types of error metrics, MAPE and RMSE. RMSE quantifies the error’s variability

Table 4. Comparison of Precision, Recall, and F1-score of the proposed method with other approaches Techniques

Cart-Knn

Figure 3. Graphical comparison for other performance metrics.

Sigma J Eng Nat Sci, Vol. 43, No. 5, pp. 1484−1494, October, 2025

Table 5. Comparison with state of art methods based on the OASIS dataset Reference

Gaussian NB, DT, RF, XGBoost, Voting Classifier, and Gradient Boost

while MAPE calculates the difference between the predicted and the actual result [49,50]. The following equation can express the following equation: (4)

(5) where N, yi and are several samples, actual and predicted output respectively. A lower value of MAPE and RMSE indicates a better prediction. Figure 5 and Figure 6 represent the optimum parametric condition of the proposed CART -KNN model.

Table 6. Computational value of F1 score for CART-KNN model for different leaf nodes value Leaf nodes

The F1 score for the train and testing dataset almost ranges the same lies from 0.25 to 0.54 for different values of no. of the leaf node shown in Table 6, while MAPE is increased for the training dataset by increasing the number of neighbors, but for the testing dataset, MAPE is reversed shown in Table 7.

Table 7. Computational value of MAPE for CART-KNN model for different neighbor value No of Neighbour

Cart-Knn

Sigma J Eng Nat Sci, Vol. 43, No. 5, pp. 1484−1494, October, 2025

Sigma J Eng Nat Sci, Vol. 43, No. 5, pp. 1484−1494, October, 2025

Figure 7 and Figure 8 show the prediction performance of the proposed hybrid model and traditional models. The proposed fused CART-KNN model improved the RMSE by 75.31%, and 74.31% while MAPE by 42.85%, and 2.20% respectively concerning traditional DT, and KNN models. Furthermore, Table 8 shows that the hybrid model is closest to the real value and has minimal swings in prediction errors.

real-world applications. To make the concept feasible, work on running time complexity reduction can be done

Nomenclature

The hybrid strategy of K nearest neighbor and prepruned classification and regression tree (CART) is suggested in this article. To overcome the overfitting problem, a pre-pruned Decision tree is applied in m-dimensional space and stores training samples of leaf nodes. Open Access Series of Imaging Studies (OASIS) datasets dataset for Alzheimer’s disease (AD) prediction have been used to test the suggested strategy, and the outcomes are compared with those of other cutting-edge methods based on OASIS datasets and machine learning models. The main conclusions are listed as follows:

2. The proposed fused CART-KNN model improved the

RMSE by 75.31%, and 74.31% while MAPE by 42.85%, and 2.20% respectively concerning traditional DT, and KNN models.

3. The validation accuracy of the proposed CART-KNN is

about 98.23% on the test data of AD, which indicates the CART-KNN classifier in the suggested study produces a more advantageous result. However, this study has also some limitations: The CART-KNN suffered in poor generalization ability due to easy overfit. New hybrid approaches based on conventional machine learning techniques can be created in the future to increase performance and make them suitable for

d(Pi,Pj) Distance between two points fault-prone classes Li Weighted value value for ith neighbor wi N no. of samples the output of ith number of samples yi Predicted output of ith samples

Data Availability Statement

Publicly available datasets were analyzed in this study. This data can be found at: https://www.kaggle.com/jboysen/mri-andalzheimers?select=oasis_cross-sectional.csv.

Data Availability Statement

The authors confirm that the data that supports the findings of this study are available within the article. Raw data that support the finding of this study are available from the corresponding author, upon reasonable request.

Conflict Of Interest

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics

There are no ethical issues with the publication of this manuscript.

Sigma J Eng Nat Sci, Vol. 43, No. 5, pp. 1484−1494, October, 2025

Statement On The Use Of Artificial Intelligence

Artificial intelligence was not used in the preparation of the article.

Share and Cite