Homnormal A comprehensive R package for testing the homogeneity of variances

Abstract

Introduction

In many scientific applications, testing the homogeneity of variances is widely employed. One of its applications, in experimental design, one of the primary assumptions of ANOVA is the homogeneity of variances of several normal populations. It is well known that violating of the assumption of homogeneity of variances impacts the type I error rate of the F-test, particularly in cases where sample sizes are unequal. Therefore, before ANOVA, it is common practice

to test the homogeneity of variances. The ANOVA F-test is also widely used in various areas [1,2]. Furthermore, assessing uniformity is crucial in various fields such as quality control in manufacturing processes, biology, agricultural production systems, and the development of educational methods. For instance, variations in population variability are of interest to biologists for several reasons, such as the investigation of adaptation mechanisms and as a measure of general diversity. [3]. A study examining the batch-to-batch

*Corresponding author. *E-mail address: eyigit@gazi.edu.tr This paper was recommended for publication in revised form by Editor-in-Chief Ahmet Selim Dalkilic Published by Yıldız Technical University Press, İstanbul, Turkey © Author. This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

variability in laboratory animals’ resistance to parasitic infestation evaluated the homogeneity of variances. In this study, rats injected with 500 larvae were analyzed to determine whether the variability in resistance differed across batches. This demonstrates that homogeneity of variance is a significant issue in biological experiments and medical research [4]. The homogeneity of variance test is used for various purposes in real estate valuation processes and market analysis studies. In particular, it is used to compare different types of real estate and sub-markets. By examining the variability levels of different types in the markets, it helps to identify which types of properties are more homogeneous and, therefore, can be evaluated more reliably [5]. The homogeneity of variance test also plays an important role in education. For example, in a study, the success scores of groups with homogeneous and heterogeneous levels of prior knowledge among students were examined to evaluate how students’ learning processes were affected [6]. A review of the literature reveals that numerous methods have been developed to test the homogeneity of variances across several normal populations. Among these, the Bartlett’s test [7] is the most widely used. This test is based on the likelihood ratio test (LRT), divided by an approximation of its mean, and it approximately follows a Chi-squared distribution. However, Bishop and Nair [8] and Hartley [9] noted that this test is only valid when the sample sizes are moderate to large and when the number of groups is not excessively high. Many studies have been conducted to identify a testing procedure that maintains the type I error rate while ensuring high statistical power. Hartley [9] proposed the F-max statistic, which represents the ratio of the maximum variance to the minimum variance among the groups. Subsequently, using the absolute differences of observations from their means, Levene [10] introduced an ANOVA-based testing strategy. Additionally, by substituting sample medians for sample means, Brown and Forsythe [11] modified Levene’s test. To control the family-wise type I error rate, Bhandary and Dai [12] suggested a test based on a Bonferroni-type adjustment process applied to the ordered p-values. While some of these tests have an exact distribution, most of them have an asymptotic distribution. As known, tests with asymptotic distribution do not give good results in small sample sizes, especially in terms of type I error. Especially in recent years, it is seen that methods based on resampling techniques have become popular because of the increase in the performance of computers. For this reason, many tests have been developed using the resampling technique. These methods based on the resampling methods, such as generalized p-value, parametric bootstrap, and computational approach are often used for solving the problem containing the interested parameters as well nuisance parameters. Using the generalized p-value approach, Liu and Xu [13] presented a test and evaluated it with Bartlett’s test. Gökpınar and Gökpınar [14] suggested a test statistic based on the computational approach test (CAT), a specific

case of parametric bootstrap, for homogeneity of variances. Chang et al. [15] suggested the CAT approach of the likelihood ratio test (LRT) and Bartlett’s test for homogeneity of variances. They compared these tests with existing tests such as Bartlette’s test, Levene’s test, Brown- Forsythe’s test, Keyes and Levy’s adjusted test [16], and Loh’s test [17]. According to the results of their study, the CAT approach of the LRT performs better than some existing tests under the normal model. Gökpınar [18] a proposed standardized likelihood ratio test (SLRT) which is not based on any resampling methods and intensive computer methods for homogeneity of variances under normality. She compared this test with Bartlett’s test and the likelihood ratio test’s Monte Carlo approach. She noted that the SLRT performs better than others. The aim of most work on methods for testing the homogeneity of variances is to develop a test with a well-controlled type I error rate and high power whenever possible. However, tests either suffer from severe inflation of the type I error rate or lose statistical power to detect heterogeneity of variances. Therefore, much work has been and is being done to improve type I error rates and the power of tests. There are some R packages such as car [19], stats [20], lawstat [21], rstatix [22], onewaytests [23] contain very few of the tests mentioned above (i.e., Bartlett’s test and Levene’s test) for homogeneity of variances. However, there is no package containing recent methods, especially resampling-based tests. As mentioned above, these tests are powerful and controlling type I error rates better than others. Because of this, our main purpose in this study is to construct a way to choose the best possible testing procedure for the homogeneity of variances. To do this, initially, we create a package that includes most powerful tests obtained in recent years and some other powerful classical tests in the literature. After that, we compare these tests according to their empirical power of tests and type I error rates to detect the best-performing test in specific situations. Thus, the researcher can easily decide which test should be used in specific situations. For this purpose, we create an R package called homnormal by taking into consideration classical tests such as Bartlett’s test (BT), Levene’s test (LT), Brown-Forsthye’s test (BFT), Bandairy Dai’s test (BDT) and resample-based tests such as generalized p-approach (GPA), computational approach test (CAT), likelihood ratio test based on CAT (LRCAT), standardized LRT (SLRT). The homnormal package is available on the CRAN. We also compare these tests according to their power of test and type I error rates with comprehensive simulation study under different situations. Thus, the researcher can easily decide which test to use in which situation. As mentioned above, the homogeneity of variances has many applications. For the sake of simplicity, in this study, we consider a one-way ANOVA model and have discussed the tests used in this field. In future studies, this package can be extended by addressing key assumptions of classical linear regression analysis, such as the homogeneity of

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

variances. In classical linear regression analysis, there have been several tests developed to test the presence of heteroscedasticity. Among them, the tests developed by White [24], Breush and Pagan [25], Goldfeld and Quandt [26] are the most well-known tests, but there are also tests recently developed by Çelik [27,28]. The rest of this study is organized as follows. Firstly, the tests for homogeneity of variances are given. Secondly, we introduce the homnormal package and demonstrate the applicability of the package using a dataset. Thirdly, a comprehensive Monte Carlo simulation study is carried out to compare the performance of the tests regarding their type I error rates and power under various conditions. This paper concludes with several final remarks. METHODS: TESTS FOR HOMOGENEITY OF VARIANCES Let Xi1,…, Xin_i, i = 1,…,k be a random sample of size ni from a normal population with E(Xij) = μi and Var (Xij) = σi2, i.e. Xij~N(μi, σi2). Let X̅i and Si2 be the sample mean and sample variance of the ith population, respectively, where

approximately F(k-1),(N-k) distribution. For a given α, the H0 is rejected if TL > F(k-1),(N-k),α). This test is called as LT throughout the paper. Brown-Forsythe’s test (BFT) Brown and Forsythe [11] developed a modification of Levene’s test by using the median instead of the mean as where is the median of group i. Thus, the Brown-Forsythe’s test is follows as (3) and . BF has where approximately F(k-1),(N-k) distribution. For a given α, the H0 is rejected if TBF > F(k-1),(N-k),α). Bhandary-Dai Test (BDT) Bhandary and Dai [12] proposed a test based on Bonferroni type adjustment procedure. The F test statistics are (4)

respectively, where N = ∑ki=1ni. The hypotheses of interest are

As mentioned in Section 1, there are many tests improved in the literature to test H0 against H1. In the following, we describe these tests. Bartlett’s Test (BT) Bartlett [7] proposed the following test statistic:

< P(2) < ... < P(2k) is obtained by sorting Pi and Pi’. For a given α, the H0 is rejected if The Generalized p-value Approach (GPA) Liu and Xu [13] proposed the generalized p-value based on the generalized test variable calculated as in (5)

(1) TB statistic has approximately χ2k-1 distribution. For a given α, the H0 is rejected if TB> χ2k-1,α. Levene’s Test (LT) Levene [10] conducted a one-way ANOVA with the dependent variable being as . The Levene’s test statistic is given by

where . E(lnUi) and V(lnUi) terms are obtained by using Monte Carlo integration. H0 is rejected if p < α. Computational Approach Test (CAT) Gökpınar and Gökpınar [14] proposed a test based on the computational approach test (CAT), which is a kind of parametric bootstrap method, for the homogeneity of variances under normality. The test statistic, TCAT, is given as

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

(6) . This test is computed using where Monte-Carlo simulations with algorithm:

7. In the case of

Standardized Likelihood Ratio Test (SLRT) Gökpınar [18] proposed a standardized likelihood ratio test by using the exact mean and variance of the likelihood ratio test statistic TLR in Eq. (8). The mean and variance of TLR are given as

7. In the case of

, H0 is rejected. Likelihood Ratio Test based on CAT (LRCAT) The likelihood ratio test for the homogeneity of variances under several normal populations is obtained as in Eq. (8). (8) . TLR has approximately χ2(k-1) diswhere tribution. The parametric bootstrap approach of this test proposed by Chang et al. [15] is computed using MonteCarlo simulations with algorithm:

5. Repeat steps 3 and 4 many times, L, and recalculate the

where Γ’(.) and Γ’’(.) are digamma and trigamma function, respectively. The standardized likelihood ratio test is obtained as follows (9) TSLR has approximately χ2(k-1) distribution. For a given α, the H0 is rejected if TSLR > χ2k-1,α. This test is called as SLRT throughout the paper. Some R packages such as car [19], stats [20], lawstat [21], rstatix [22], onewaytests [23] contain the tests for homogeneity of variances. Table 1 shows which tests are included in which packages. As can be seen from Table 1, there are only a few tests in the packages, although most are not new tests. Furthermore, there is no package that also contains recently proposed tests of homogeneity of variances. Onewaytests package used for one-way tests in independent groups design contains only BT, LT, and BFT. As mentioned in the introduction section, testing the homogeneity of variances is used in many fields, apart from an assumption used to apply the ANOVA-F test. For this reason, it is important to develop a package that includes many tests used in testing the homogeneity of variances. We wish to continue with the plan to build a package that contains tests not found in CRAN.

Table 1. Comparison of R packages including homogeneity of variances tests Tests R packages

Lrcat

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

The Usage Of Homnormal Package

The package including several functions provides to perform several tests for homogeneity of variances. These functions are identified by their names’ initials, which are listed in the preceding sections. In this section, we use a data set to show how to use the homnormal package. Example: As an example, FH_data is presented in the package. The data related to the survival times of patients was collected from 4 hospitals, which was a part of the data by Fleming and Harrington [29]. Our purpose is to test for the homogeneity of variances for this data. # Call the homnormal package library(homnormal) # Call the huxtable package library(huxtable) # print observations of the FH_data x1=FH_data$SurvivalTime x1 #> [1] 105 266 227 66 24 5 155 54 58 64 15 147 42 305 92 30 82 265 237 #> [20] 208 147 x2=FH_data$HospitalNo x2 #> [1] 1 1 1 1 2 2 2 2 3 3 3 4 4 4 4 4 4 4 4 4 4

The homnormal package includes eight tests for testing the homogeneity of variances.

Brown-Forsythe’s Test (BFT): Brown_Forsythe(...) The Brown_Forsythe function in the homnormal package is used to perform the Brown-Forsythe’s test. BFT = Brown_Forsythe (x1, x2, alfa = 0.05, table = TRUE, graph = ”centerized”)

In this code, alpha is the level of significance to assess the statistical difference. Default is set to alpha = 0.05. table is a logical value indicating whether the analysis results will be displayed in tabular form. Default is table=TRUE. Graph is a logical value for printing a boxplot of data. Default is set to graph=”none”. The alternative arguments of graph are “raw”,”centered”. “raw” gives a boxplot of raw data. “centered” gives a boxplot of centered data around means of group’s. When this code is run in R Programme, the following Figure 1 and Table 2 output are obtained. Box plots provide preliminary information to understand whether the variances of groups are heterogeneous or not, but it is a visual assessment method, not a definitive test. The height of the box in a box plot is an indicator of the variability within a group. If there are significant differences in box heights between groups, this indicates that variances may be heterogeneous. Compare the “whisker” lengths of each box plot. Whiskers usually show minimum and maximum values. If the whisker lengths differ between

Tablo 2. Brown-Forsythe’s Test Result for Homogeneity of Variances Group no

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

groups, this may also indicate variance differences. In light of this information, it can be said that there is heterogeneity between the variances of the groups. But to be more precise, we need to look at the test results. According to these results, the null hypothesis of the homogeneity of variances is not rejected (BFT=1.88, p-value=0.17) since the p-value is not smaller than 0.05. In other words, it can be said that the variances of the groups are homogenous at 5% significance level. The users who would like to use the statistics in the output of their programs can use the following codes. The BT, LT, BDT, GPA, CAT, LRCAT, and SLRT tests are also available as functions in the homnormal package. These tests are shown below.

When this code is run in R Programme, the following Table 4 output are obtained. The null hypothesis of the homogeneity of variances is not rejected (BT=3.06, p-value =0.38) since the p-value is not smaller than 0.05. According to the results of Bartlett’s test, it can be said that the variances of the groups are homogenous at 5% significance level.

Levene’s Test (LT): levene(...) The levene function in the homnormal package is used to perform the Levene’s test. LT = levene (x1, x2, alfa = 0.05, table = TRUE, graph = “none”)

When this code is run in R Programme, the following Table 3 output are obtained. Similar to BFT, according to the results in Table 3, the null hypothesis of the homogeneity of variances is not rejected (LT=2.16, p-value =0.13) since the p-value is not smaller than 0.05. In other words, according to the results of Levene’s test, it can be said that the variances of the groups are homogenous at 5% significance level. Bartlett’s Test (BT): bart(...) The bart function in the homnormal package is used to perform the Barlett’s test. BT = bart(x1, x2, alfa = 0.05, table=TRUE)

Bhandary-Dai’s Test (BDT): bdai(...) The bdai function in the homnormal package is used to perform the Bhandary-Dai’s test. BDT = bdai(x1, x2, alfa = 0.05, table = TRUE)

When this code is run in R Programme, the following Table 5 output are obtained. The null hypothesis of the homogeneity of variances is not rejected (BDT=11.46, p-value =0.66) since the p-value is not smaller than 0.05. In other words, according to the results of Bhandary-Dai’s test, it can be said that the variances of the groups are homogenous at 5% significance level. Generalized p-value Approach (GPA): genp(...) The genp function in the homnormal package is used to perform the Generalized p-value approach. GPA = genp(x1, x2, alfa = 0.05, m = 5000, table = TRUE)

When this code is run in R Programme, the following Table 6 output are obtained. The null hypothesis of the homogeneity of variances is not rejected (GPA=2.10, p-value =0.50) since the p-value is not smaller than 0.05. In other words, according to the

Table 3. Levene’s test result for homogeneity of variances Group no

Table 4. Bartlett’s Test Result for Homogeneity of Variances Group no

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

Table 5. Bhandary-Dai’s test result for homogeneity of variances Group no

results of generalized p-value approach, it can be said that the variances of the groups are homogenous at 5% significance level. Computational approach test (CAT): Cat_GG(...) The Cat_GG function in the homnormal package is used to perform the computational approach test. CAT = Cat_GG(x1, x2, alfa = 0.05, m = 2000, table = TRUE)

is not smaller than 0.05. In other words, according to the results of likelihood ratio test based on CAT, it can be said that the variances of the groups are homogenous at 5% significance level. Likelihood Ratio Test Based on CAT (LR-CAT): Cat_ LR(...) The Cat_LR function in the homnormal package is used to perform the likelihood ratio test based on CAT.

When this code is run in R Programme, the following Table 7 output are obtained. The null hypothesis of the homogeneity of variances is not rejected (CAT=16.35, p-value =0.24) since the p-value

LR_CAT = Cat_LR(x1, x2, alfa = 0.05, m = 5000, table = TRUE)

Table 6. Generalized p-value aproach result for homogeneity of variances Group no

Table 7. Compuational aproach test result for homogeneity of variances Group no

Table 8. Likelihood ratio test result for homogeneity of variances Group no

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

Table 9. Standardized likelihood ratio test result for homogeneity of variances Group no

The null hypothesis of the homogeneity of variances is not rejected (LR_CAT=5.85, p-value =0.28) since the p-value is not smaller than 0.05 (Table 8). Standardized Likelihood Ratio Test (SLRT): slrt(...) The slrt function in the homnormal package is used to perform the standardized likelihood ratio test. SLRT = slrt(x1, x2, alfa = 0.05, table = TRUE)

The null hypothesis of the homogeneity of variances is not rejected (SLRT=3.73, p-value=0.29) since the p-value is not smaller than 0.05. The p-values of all tests indicate that the tests do not reject the null hypothesis of the homogeneity of variances at a nominal level 0.05. That is, each of the tests suggests that variation in survival times of patients is not significant among 4 hospitals. This study includes an example from survival analysis, where the sample sizes in each group are 4, 4, 4, and 10. These are quite small, and if there is genuine heterogeneity among the groups, it is crucial to use a test that performs well in detecting such differences. According to our simulation study, the LRCAT and SLCAT tests demonstrated excellent performance for small sample sizes. Therefore, it is particularly important to consider the results of these tests in this context (Table 9).

Simulation Study

In this section, we investigate the performance of the BT, LT, BFT, BDT, GPA, SLRT, CAT, and LRCAT tests by estimating the type I error rates and powers. To determine the type I error rates and powers of the tests, 5000 replications were performed. Additionally, the p-values of the CAT and LRTCAT methods were estimated using Monte Carlo simulations with m=5000. Sample size combinations include cases where the sample sizes are equal and cases where they are unequal. Random samples were generated from a normal distribution with parameters μi and σi2 across different groups and sample sizes. Without loss of generality, μi; i = 1,…, k was set to 0. For the nominal level of 0.05, the estimated type I error rates of all tests are provided in Table 10. It is seen from Table 10, that while the LT tends to be liberal, the BFT test tends to be conservative in cases of

equal and unequal small sample sizes. The estimated type I error rates of other tests are very close to the nominal level of 0.05 for all considered cases regardless of sample sizes and group numbers. For the specified nominal level of 0.05, the estimated powers of all tests were presented in Table 11Table 18. In addition, for certain cases in the tables, the power values of the tests are also presented in the following graphs. Accordingly, the first graph corresponds to k=3 with σ2 = (1,3,9), the second graph to k=5 with σ2 = (1,1,3,9,9), the third graph to k=7 with σ2 = (1,1,3,3,3,9,9), and the final graph to k=9 with σ2 = (1,1,1,3,3,3,9,9,9). Since we take the combinations of sample sizes as equal and unequal sample sizes, tables are obtained for each group according to both equal and unequal sample sizes. This has been considered when interpreting the simulation results. While interpreting the test powers, the LT test was excluded because its estimated sizes exceeded 6%, as shown in Table 10. As seen from Table 11 for k=3 and equal sample sizes, in cases of small sample sizes, the performances of the BDT, SLRT, and LRCAT tests are higher than the others. When the sample sizes increase, these tests have still the highest power, but the BT test approaches the powers of them. As seen from Table 12 for k =3 and unequal sample sizes, the differences between the performances of the tests become more distinctive. In cases of small sample sizes, the powers of the SLRT and LRCAT tests are remarkably higher than those of the others. When the sample sizes increase, the CAT test gives better results as well as these tests. It is expected that in equal and unequal sample sizes, as the difference between variance parameters increases, the power values of all the tests increase and close to each other. As seen from Table 13 for k=5 and equal sample sizes, the powers of the SLRT and LRCAT tests are higher than others in cases of small sample sizes. As sample sizes increase, the BT test as well as these tests have higher power values than others. As seen from Table 14 for k=5 and unequal sample sizes, the powers of the SLRT and LRCAT tests are remarkably higher than those of the others. As seen from Table 15- Table 18, the same pattern is valid for other k. Chang et al. [15] compared LRTCAT with tests other than SLRT in their study. According to their simulation results, LRTCAT outperformed the other tests. Similarly, Jafari and Shaabani [30] stated in their study

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

Lrcat

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

Table 11. Powers of all tests for k=3 with equal sample sizes σ2

Lrcat

Table 12. Powers of all tests for k=3 with unequal sample sizes σ2

Lrcat

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

Table 13. Powers of all tests for k=5 with equal sample sizes σ2

Lrcat

Table 14. Powers of all tests for k=5 with unequal sample sizes σ2

Lrcat

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

Table 15. Powers of all tests for k=7 with equal sample sizes σ2

Lrcat

a1 = (1,1,2,2,2,4,4); a2 = (1,1,3,3,3,9,9); a3 = (1,1,4,4,4,16,16)

Table 16. Powers of all tests for k=7 with unequal sample sizes σ2

a1 = (1,1,2,2,2,4,4); a2 = (1,1,3,3,3,9,9); a3 = (1,1,4,4,4,16,16)

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

Table 17. Powers of all tests for k=9 with equal sample sizes σ2

Lrcat

a4 = (1,1,1,2,2,2,4,4,4); a5 = (1,1,1,3,3,3,9,9,9); a6 = (1,1,1,4,4,4,16,16,16)

Table 18. Powers of all tests for k=9 with unequal sample sizes σ2

Lrcat

a4 = (1,1,1,2,2,2,4,4,4); a5 = (1,1,1,3,3,3,9,9,9); a6 = (1,1,1,4,4,4,16,16,16)

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

Figure 2. Power values of the tests for k=3,5,7,9 with equal sample sizes.

that SLRT is a good alternative for testing the homogeneity of variances compared to the tests they examined, excluding LRTCAT. Our literature review indicates that no study has compared SLRT and LRTCAT simultaneously. However, previous research shows that LRTCAT performed better than the methods it was compared against, and similarly, SLRT outperformed the methods it was tested against. In our simulation study, both SLRT and LRTCAT were considered alongside other tests. The results demonstrate that both tests performed better than the alternatives, confirming that our findings align with those of previous studies. The widely used BT test, despite its popularity, exhibited notable limitations under conditions of small and unequal sample sizes, confirming its sensitivity to such scenarios as highlighted in [13,15,18]. This simulation study highlights the importance of alternative approaches such as SLRT and LRCAT in this context. These tests have been observed to give much better results than other

tests, especially at small sample sizes. Since data collection is challenging due to various constraints, especially in real-life applications, the practical implications of these findings are getting more significant. For instance, in survival analysis, sample sizes are often limited by the availability of specimens, patient recruitment difficulties, or high costs associated with longitudinal studies. Similarly, in environmental studies, obtaining measurements from remote or hazardous locations can restrict sample sizes. In clinical trials, ethical considerations and the scarcity of rare disease patients often result in small sample sizes. In genetics, experiments on rare genotypes or high-throughput sequencing can also face limitations due to the cost and complexity of data collection. These scenarios underscore the importance of using tests like the SLRT and LRCAT, which perform robustly even under small and unequal sample sizes, as supported by our findings and consistent with prior research.

Sigma J Eng Nat Sci, Vol. 44, No. 3, pp. 1619−1634, June, 2026

Share and Cite