An amalgamation of crisp and fuzzy quantile regression model

Abstract

Keywords: Fuzzy Regression; Quantile Regression; Fuzzy Data; Linear Programming; Simplex Procedure

Introduction

Regression is one of the most important statistical measure for applied research and is applied to evaluate the relationship between response variable and explanatory variables. It also make some assumptions about the shape

or distribution of residuals like heteroscedasticity, auto-correlation and multi-collinearity. If these assumptions are present in the model then the results of the estimates become misleading. An ordinary least squares (OLS) technique is used for estimating the standard regression parameters.

*Corresponding author. *E-mail address: aliakgul00727@gmail.com This paper was recommended for publication in revised form by Regional Editor Hijaz Ahmad Published by Yıldız Technical University Press, İstanbul, Turkey Copyright 2021, Yıldız Technical University. This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 1−10, February, 2024

The output of the typical linear regression methodology is the mean average between a set of explanatory variables, while response variable is called conditional mean whose function is stated as 𝐸(𝑌/𝑋). As one might be interested in conditional distribution of 𝑌 by observing the relationship at different points but the linear regression technique only offers a limited view of the relationship. Therefore another technique called quantile regression is used in this situation which has capability to describe the relationship in the conditional distribution of 𝑌 at different points. Quantile regression falls in a non parametric approach as non parametric is not based on any assumptions, similarly quantile regression also makes no assumption about the distribution of the residual or error term. Quantile regression permits us to study the effect of different predictors on various quantiles of the response variable, so that it provides a comprehensive picture of the relationship between response and explanatory variable. It is more vigorous to outliers than the casual least squares regression. Measurement results and observational data are often not precise numbers but may be less or more precise, it is also called fuzzy. The idea of assembling sets as generalized sets was firstly published in [1] Menger, and later Zadeh [2] introduced the term ‘fuzzy set’ [3]. All types of data which cannot be shown as precise numbers or cannot be categorized precisely are said to be non-precise or fuzzy. The data in the form of linguistic or verbal descriptions like low temperature, high speed and high blood pressure etc are the examples of fuzzy. Fuzzy set theory is the most powerful tool to describe the process of uncertainty and imprecise information which exist in real world, and fuzzy regression is an important research topic which can be used to fulfill predicting by establishing the functional relationship between fuzzy variables. There are a many studies have been done to merge statistical techniques and fuzzy set theory. Li and Zeng [4] modified fuzzy regression based on least absolute deviation which is also known as median regression. They introduced a new distance measure between fuzzy number and merge with median regression or LAD and then parameters are considered to be a trapezoidal fuzzy number, Otadi [5] discussed the case fuzzy input and fuzzy coefficients in fuzzy polynomial regression, Choi [6] applied LAD (least Absolute Deviation) estimation technique to generate fuzzy regression analysis, Tian and Zhu [7] used EM-Algorithm to derive a new procedure for estimating linear quantile regression, Das and Ghosal [8] used Bayesian method to solve simultaneous quantile regression. In this approach, quantile function is obtained by inserting the prior distribution on monotone increasing function, Kim et al. [9] considered a single case fuzzy input-output with LAD (Least Absolute Deviation) regression, D'Urso [10] applied least square procedure for estimating the parameters of different fuzzy linear regression, where the model is based on fuzzy output with fuzzy input and Tanaka and Guo [11] recommended a technique to evaluate the efficiency of the model with fuzzy input and

output. Similarly, Rasheed and Mustafa [12] done comparative study between simple linear regression, quantile regression and spline regression and Khalid and Mustafa [13] developed a quantile regression model using B-spline and penalties where she used method least absolute deviation (LAD) and include penalty term in the objective function. It has been observed that most of the the researchers relates the fuzzy set theory with the linear regression, logistic regression or narrates to the statistical models and named them fuzzy linear regression. While the work done on quantile regression, mostly includes the the quantile regression with splines and most of them with Bayesian approach. Therefore, this study has attempted to merge the crisp quantile regression with fuzzy set theory which motivates the combination of both theories in order to obtain more reliable models which could be successfully applied in many practical studies of quantile regression models in real world application when data is not precise or exact for example the proposed model can be used in bio medical research study to understand the relationships among different fuzzy variables outside of the conditional mean of the response, in the field of epidemiology for the risk prediction and to examine the effects of growth trajectories and similarly they can be used in weather forecasting, wind speed forecasting and estimation of food expenditures. The rest of the paper is organized as follows, some basic concepts of fuzzy set theory and its related components and some techniques which are used for present work are discussed in Section 2. In Section 3, methodology of proposed work is discussed. In Section 4, we have discussed the numerical computations and some concluding remarks are given in Section 5.

Preliminaries

In this section, some basic definitions related to present work are discussed. Quantile Regression and Weighted Absolute Function Quantile regression is a semi parametric approach because it avoids the assumptions regarding the parametric distribution of the error term. Quantile regression contributes much additional information for the response variable. In OLS, the estimates of linear regression is the conditional mean function, while the estimates of quantile regression is based on conditional quantile function 𝑄(𝑌/𝑋). The special case of quantile regression is known as Least Absolute Deviation Regression and simply abbreviated as LAD. LAD studies the connection between the explanatory and response variable using the conditional of median function 𝑄0.5(𝑌/𝑋) where the median equals to the 50𝑡ℎ percentile of the observed or experimental distribution. The quantille (𝜃 → [0,1]), splits the data into proportions 𝜃 for under estimation and (1 − 𝜃) for over estimation. If 𝜃 = 0.5 then it becomes Median regression.

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 1−10, February, 2024

The estimation of parameter 𝛽′𝑠 in these three models are different. If 𝜖𝑖 is the model prediction error, then OLS minimizes as,

Median regression is estimated by least absolute deviation (LAD) approach, which minimizes

In quantile regression model (QRM), the model and its conditional mean function is given as,

where the 𝜏𝑡ℎ quantile of 𝜖𝑡 is zero, and (𝜃 ∈ [0,1]) quantile. Quantile regression minimizes the sum of the unsymmetrical penalties 𝜃|𝜖𝑡| for under prediction and (1 − 𝜃)|𝜖𝑡| for over prediction. The estimator of quantile regression for any 𝜃 belongs to [0,1] minimizes the objective function;

where 𝐼{𝜖⩾0} and 𝐼{𝜖<0} are the check or indicator functions, defined by Yu et al. [14]. The equation

is a non-differential function and the loss function is a piece-wise linear. Therefore,it is a linear programming problem and its computation is quite difficult so it requires linear programming method for example simplex method to minimize it, which is an appropriate method to acquire a solution in a finite number of iterations ( see for example [15]). An alternative simpler notation of 𝜌𝜃(𝜖) is where 𝜖+ represents 𝐼{𝜖⩾0}|𝜖𝑡| and 𝜖− represents 𝐼{𝜖<0}|𝜖𝑡| . The objective function is represented as,

Fuzzy Logic and Sets Fuzzy logic is a generalization of classical logic. It is a mathematical technique to handle with the uncertainty, where it model those problems which are utilizing the imprecise data. As fuzzy logic deals with linguistic terms by assigning the suitable membership degree. So it can say that fuzzy frequently reports with uncertainty, imprecision and vagueness. In this section we recall some notations and preliminary concepts about fuzzy sets and numbers (see [3]). The mathematical definition of fuzzy sets can be defined as; Definition 1 A fuzzy set is a collection of ordered pairs, Y be universe of discourse and y is specific element of Y, so the fuzzy set A can be written as,

where 𝜇𝐴̃(𝑦) showing a degree of membership of element y with 𝜇𝐴̃(𝑦) → [0,1] and each ordered pair (𝑦, is known as singleton. 𝜇𝐴(𝑦)) ̃

Linguistic Terms Linguistic variables are essential for fuzzy sets. Basically, these are the words that are converted into numeric values by using fuzzy set theory. It is very helpful for dealing with vague and inadequate information as a first step to overcome ambiguity.

Membership Function Membership is a degree of truthfulness of an input. The degree always lies between 0 to 1. It is denoted by 𝜇𝐴(𝑦), ̃ where 𝐴̃ is a fuzzy set and 𝑦 ∈ 𝑌. This functions permits to draw fuzzy sets graphically, where x-axis has a values of universal set and y-axis has values of membership on an interval [0,1].There are different types of membership functions and some of them that at are most commonly used are triangular, trapezoidal and gaussian membership function. Definition 2 Suppose x be a subset of X, a membership where 0 ≤ function of a fuzzy set 𝐴̃ is symbolized as 𝜇𝐴(x), ̃ ≤ 1. The value of 𝜇 (x) is called the degree of mem𝜇𝐴(x) ̃ 𝐴̃ ̃ bership x ∈ X which is defined on a fuzzy set 𝐴 If 𝜇𝐴̃(x) is zero, it will shows that it is not a member of fuzzy set 𝐴̃ and if 𝜇𝐴̃(x) = 1 then is shows that it a fully member of given fuzzy set. Fuzzy Number Definition 3 A fuzzy number B is called a fuzzy subset of R, then the membership function μB(x) must obey the following standards, 1. 𝛼 − 𝑐𝑢𝑡: 𝜇𝐵(𝑥) → [0,1]

2. The set of 𝛼 − 𝑐𝑢𝑡 must have such elements of x whose

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 1−10, February, 2024

3. There exist x, i.e. 𝜇𝐵(𝑥) = 1.

4. 𝜇𝐵(𝑥) is a piece-wise continuous membership function. Definition 4 If x belongs to the set X, and all the elements of X has non-zero membership magnitude in B, i.e,

algorithms ever invented, and it is still the standard method employed on computers to solve optimization problems. The given LPP is easy to solve when the variables involve in the problem and their respective constraints are lesser in number but when the number of variables and constraints become larger, it is not possible to solve it manually, so one can overcome this complexity by using appropriate statistical software.

𝜇𝐵(𝑥) is membership function defined on a closed interval [0,1]. Theorem 1 Let A = (al, am, ar) and B = (bl, bm, br) are two triangular fuzzy numbers in a universe of discourse X, then 1. 𝐴 ⊕ 𝐵 = (𝑎𝑙, 𝑎𝑚, 𝑎𝑟) + (𝑏𝑙, 𝑏𝑚, 𝑏𝑟) = (𝑎𝑙 ⊕ 𝑏𝑙, 𝑎𝑚 ⊕ 𝑏𝑚, 𝑎𝑟 ⊕ 𝑏𝑟) 2. −(𝑎𝑙, 𝑎𝑚, 𝑎𝑟) = (𝑎𝑟, −𝑎𝑚, 𝑎𝑙)

Fuzzy Regression The term fuzzy regression analysis was introduced by Tanaka and Guo [11]. Possibilistic and least-squares are two main approaches of fuzzy regression models. The idea of best fit in both approaches includes the problem which is associated with the funtion optimization. Particularly, in possibilistic method, this functional takes the form of a measure of the spreads of the estimated output, either as a weighted linear sum involving the estimated coefficients in linear regression, or as quadratic form in the case of exponential possibilistic regression. Secondly in the standard least-squares approach, the functional to be minimized the distance between the observed and estimated outputs. This reduces to a class of quadratic optimization problems and constrained quadratic optimization D′Urso [10].

Linear Programming Problem The problem of optimizing the linear function subject to its linear restriction or constraints, where the motive is to find the vector ∈ 𝑅. The constraints may be in the form of equalities or inequalities such as (=, ≤, ≥) respectively. Let 𝑧 be the expression which is being minimized or maximized is called an objective function, and let the variable 𝑥𝑖, (𝑖 = 1,2, . . . , 𝑛) involved in the problem are called decision variables. Each line of constraints ended with the constant 𝑑𝑖. If the set of variables 𝑥𝑖 fulfills all the restrictions then it is called feasible state and the set of these points in this state is known as feasible region. The standard linear programming problem can be expressed as,

where 𝑥𝑖 ≥ 0 𝑎𝑛𝑑 𝑖 = 1,2, . . . , 𝑛. The above linear programming problem can be solved by using the simplex method which is one of the most useful and efficient

Fuzzy Quantile Regression In this study we have introduced quantile regression in fuzzy logic which gives more detailed and precise results whenever the information is inexact or imprecise. Fuzzy Linear Regression We have considered the model which is represented as, (1) where 𝑌𝑖̃ , 𝐴̃ and 𝑋𝑘 are triangular fuzzy numbers. The above model in matrix form is given as,

In simpler form the above model can be expressed as, (2) By applying the method of least square separately we can get the left, middle and right of the parameter estimates which are given below; (3) (4) (5) Proposed Models We have considered two different cases for the illustration of our proposed technique, firstly when the input and output are taken as fuzzy and secondly, when the input and output are taken as fuzzy but the parameters are crisp. Case-I: Fuzzy output, fuzzy input and fuzzy parameter (FFF) For this case we considered the following model, (6) ̃ and 𝑋𝑘 ̃ are triwhere 𝜃=Specified Quantile and 𝑌𝑖 ̃ 𝜃, 𝐴𝜃 angular fuzzy numbers.These numbers are represented as,

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 1−10, February, 2024

Subject to the constraints, (16) For more simplicity, the problem of optimization for the above equations can be converted into matrix form, such as By using definition (4), equation (6) can be written as, (7) where 𝐿−1(𝛼) = Inverse Shape function, 𝑒 = spread of fuzzy number and for instance 𝛼= 0 is used ( see for example [11]). So equation (6) can be written as,

(9) To minimize the difference between observe and estimated fuzzy numbers we can decompose the above equation into three equations(see for example [16]) and are given as, (10) (11)

where = Vectors of ones, 0= Vectors of zeros, 𝐼= Identity matrix, 𝑋𝑖𝑗=Triangular fuzzy set of independent variable, 𝑎𝑖𝑗= left, middle and right fuzzy number parameter, Similarly for equation (11), linear programming problem to obtain the parameter of the triangular fuzzy number is given as,

(12) To find the parameter of the left triangular fuzzy number we have to minimize the equation (10), for this purpose linear programming problem is given as follows, (13)

where l represents indicator function. So the equation (10) becomes,

And for equation (12), linear programming problem is represented as, (14)

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 1−10, February, 2024

(29) Case-II: fuzzy output, fuzzy input and crisp parameter (FFC) The model of quantile regression for fuzzy output, fuzzy Input and crisp parameter (FFC)is represented as,

The problem of minimization in quantile regression algorithm is given as, (30)

(23) where 𝜃=Specified Quantile, 𝑌̃𝑖𝜃 , and 𝑋̃𝑘 are triangular fuzzy numbers and 𝑎𝑖𝜃 is a crisp parameter, such as

To reach the linear programming problem (LPP), we have solved the equation (30) and simplified by considering the model up to two parameters𝑎0𝜃 and 𝑎1𝜃, so the obtained form is given as (31)

From the definition (4), membership functions of 𝑌̃𝑖𝜃 and 𝑋̃𝑖𝑗 are represented as, (24)

where 𝜀𝑖 = 𝑌𝑖𝐷 − 𝑎0𝜃 − ∑ 𝑎1𝜃𝑋𝑖𝑗𝐷 As the above equation is non differential and it is a problem of linear programming, so we can write the above equation into a simpler form, such as (32) where 𝜀+ represents the indicator function 𝐼{𝜀≥0}|𝜀𝑖| and 𝜀 represents the indicator function 𝐼{𝜀<0}|𝜀𝑖| . The objective function for LPP is represented as −

To find the crisp parameters 𝑎0𝜃 , 𝑎1𝜃 , . . . , 𝑎𝑘𝜃 we converted the fuzzy values 𝑌̃𝑖 = (𝑌𝑙, 𝑌𝑚, 𝑌𝑟), and 𝑋̃𝑖𝑗 = (𝑋𝑖𝑗𝑙 , 𝑋𝑖𝑗𝑚 , 𝑋𝑖𝑗𝑟) into crisp values by using the method of defuzzification. Let 𝑋𝑖𝐷 and 𝑌𝑖𝐷 be the defuzzified values of the fuzzy ̃ and 𝑌𝑖 ̃ respectively. There are numerous observation 𝑋𝑖𝑗 methods have been suggested in literature for defuzzification, here we are using the centroid method to defuzzify the values of 𝑋̃𝑖𝑗 and 𝑌̃𝑖 . The formula for defuzzification are represented as, (26)

Estimation Of Error And Evaluation Of Models

The errors of both models given in equation (6) and (23) are computed by taking the difference between observed and expected values of fuzzy numbers 𝑌̃𝑖 and 𝑌̃𝑖 and are represented as, (35) The evaluation of the proposed models are done by taking the advantage of Akaike information criterion (AIC). AIC is

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 1−10, February, 2024

a tool which is used to check and evaluate the quality of the model. Minimum the value of AIC shows the correctness of the model. The formula to compute the AIC is given as,

By inserting the numerical values of coefficients, the estimated model at 25𝑡ℎ quantile is represented as,

where 𝑙𝑛(𝐿)̂ is the maximum of likelihood function of the model.

Numerical Computation To illustrate the theoretical results and obtain efficiency of our proposed regression models we have considered the data set of a simulation study which is originally used by Chung [17]. The proposed linear programming problem (LPP) is consist of complex algorithm and it cannot be solved manually, therefore R-software is used to compute the model. The R-package of the proposed study is created under the latest version of R Software. The name of the proposed R-package is “FuzzReg”, where the title of package is Fuzzy Regression, the version of the package is 0.1.1. The package “FuzzReg” give the estimates of Fuzzy Linear regression (FLR) and fuzzy quantile regression (FQR). The command for Fuzzy linear regression is created on the least square method while the command for fuzzy quantile regression is made as per the proposed technique. The details of programming with codes are available in appendix section. Case-I In this case we considered the model of quantile regression when the input and output are taken as fuzzy and is given in equation (6). The coefficients of the this model are estimated using the linear programming problem by the equation (10), (11) and (12) at 0.25, 0.5, 0.75 quantile such as 𝜃 = 0.25,0.5,0.75). The data set is taken from the research paper of Chung [17] to explain and compare theoretical results of proposed models. The data contains one response triangular fuzzy number 𝑌̃𝑖 and two explanatory triangular fuzzy numbers 𝑋̃𝑖1 and 𝑋̃𝑖2.

Estimation Of Models At Different Quantiles

For one response and two explanatory fuzzy numbers the model at 𝜃 = 0.25 (at 25𝑡ℎ quantile) using equation (6) is given as, (37) where

(38) By following the same steps we also estimated the model at 50𝑡ℎ quantile and is given as (39) Similarly estimated model at 75𝑡ℎ is represented as, (40)

Estimated Model Of Fuzzy Linear Regression Case-I The model of fuzzy linear regression in the case of fuzzy input, fuzzy Output and fuzzy parameter is given as,

Case-II We considered the quantile regression model in this case when we have fuzzy input, fuzzy out put and crisp parameters.The estimation of the model given in equation (23) is done by using linear programming problem in equation (33) and (34). The data contains triangular fuzzy number i.e. 𝑌̃𝑖𝜃 = (𝑌𝑙𝜃, 𝑌𝑚𝜃, 𝑌𝑟𝜃) and 𝑋̃𝑖𝑗 = (𝑋𝑖𝑗𝑙 , 𝑋𝑖𝑗𝑚 , 𝑋𝑖𝑗𝑟). In order to find the crisp parameters (𝑎0, 𝑎1, 𝑎2) we have converted the fuzzy observations into crisp values. Centeroid method given by equations (26) and (27) is used to defuzzify the fuzzy observations.

Estimation Of Models At Different Quantiles

The model for one response and two explanatory fuzzy numbers at 25𝑡ℎ quantile is given as, (42)

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 1−10, February, 2024

The intercept and the slopes of 𝑋𝑖𝑗 are computed using the equation (33). The estimated model at 25𝑡ℎ quantile is represented as,

2. The value of AIC for proposed fuzzy quantile regression

model is less than AIC value for fuzzy linear regression in both cases, which implies the better performance of our proposed models.

(43) Similarly the estimated mode at 50𝑡ℎ quantile is represented as, (44) and estimated model at 75𝑡ℎ quantile is,

Table 1. Error and AIC of Fuzzy Linear Regression and Fuzzy Quantile Regression for Case-I Models Fuzzy Quantile Regression

Estimated Model of Fuzzy Linear Regression Case-II The model of fuzzy linear regression in case of fuzzy input, fuzzy output and crisp parameter, in terms of two input and one output fuzzy number is shown as,

and after estimating the parameters, estimated fuzzy linear regression model is given as, (46)

Comparison Of Fuzzy Quantile And Fuzzy Linear Regression

The evaluation and comparison between existing and proposed models are made by error estimation. Error estimation is done by deviating the observed values by estimated values as given in the equation (35). As fuzzy linear regression estimates a single line over the data while fuzzy quantile regression estimate multiple lines at specified quantile 𝜃. For the sack of comparison, we have compared error of fuzzy quantile regression at 75𝑡ℎ quantile with fuzzy linear regression. The error of fuzzy linear regression for the model given in Case-I is obtained as 3135.449 while the error of fuzzy quantile regression for the same model is obtained as 145.9895. In case-II, the errors are attained as 297.1842 and 51.84444 for fuzzy linear regression and fuzzy quantile regression respectively. It can be clearly seen that in both cases, proposed fuzzy quantile regression gives less error as compared to fuzzy linear regression and it can be concluded that fuzzy quantile regression performs better then the conventional fuzzy linear regression. We also extended our comparison by using Akaike information criterion (AIC) given in section (Estimation of Error and Evaluation of Models). The more detail of errors

Table 2. Error and AIC of Fuzzy Linear Regression and Fuzzy Quantile Regression for Case-II Models

Conclusion

This research study mainly concerns with combination of quantile regression and fuzzy set theory to obtain fuzzy quantile regression. Fuzzy regression analysis is considered as extensively used technique in the atmosphere of fuzzy. It is an emerging zone for research now a days. Quantile regression is also an important methodology for response variables. The situation in which the assumption of linear regression becomes fail, the quantile regression assists to produce required results. We have proposed two models of quantile regression with various combinations of input-output variables by using triangular fuzzy number.The proposed models are then converted into a linear programming problem (LPP) and are solved by simplex method, taking triangular fuzzy data set. Empirical work shows evidence in the favor of proposed work. The error and AIC of each proposed model are less than the error and AIC of fuzzy regression which demonstrates that the proposed models are best fitted models and provides excellent assistance to elegantly solve structural equation models in fuzzy environment. Empirical researchers can take advantage of proposed fuzzy quantile regression models ability to examine the impact of predictor variables on response distribution in uncertainty modeling. Other research work can be introduced in the framework of present approach to fuzzy quantile regression where any combination of the inputs, outputs and parameters could be fuzzy or type-2 fuzzy.

Sigma J Eng Nat Sci, Vol. 42, No. 1, pp. 1−10, February, 2024

Data Availability Statement

The authors confirm that the data that supports the findings of this study are available within the article. Raw data that support the finding of this study are available from the corresponding author, upon reasonable request.

Conflict Of Interest

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics

There are no ethical issues with the publication of this manuscript.

Share and Cite