Performance of imputation techniques A comprehensive simulation study using the transformer model

Abstract

Keywords: ARIMA; GARCH; Kalman Filtering; Missing Data; Monte Carlo Simulation

Introduction

In time series analysis, handling missing data is a critical challenge, as the accuracy and reliability of the results depend heavily on the effectiveness of imputation methods. These methods not only address gaps in the data but also improve the ability to predict future values. Traditional models like the Autoregressive Integrated Moving Average (ARIMA) [1] and Generalized Autoregressive Conditional Heteroskedasticity (GARCH) [2] have been extensively used for forecasting and are well-established in time series analysis. For instance, ARIMA is effective in modeling time series data by predicting future values based on past observations, but its performance can be compromised in the presence of missing data [3]. Similarly, GARCH models, known for their ability to model volatility, also struggle with incomplete datasets, potentially leading to biased estimates and inaccurate forecasts [2]. To address these limitations, advanced imputation techniques have been developed. Kalman filtering, a state-space model, has been successfully applied to estimate missing values by leveraging dynamic linear models [4]. Moreover, recent research has explored even more sophisticated imputation methods, such as the generalized m-parameter Mittag-Leffler function, which has shown promise in handling complex differential and integral equations [5]. Despite these advancements, there is still a need for a systematic evaluation of how these methods perform in different scenarios, particularly when integrated with newer models like Transformers, which have shown significant potential in time series analysis [6].

Related Work

This section presents a comprehensive review of existing studies on imputation techniques and their application in time series analysis. ARIMA and GARCH models are widely used in time series forecasting due to their robustness and efficacy in handling various types of data. These models have been extensively applied across multiple domains. For instance, [7] employed GARCH models to capture the volatility in energy markets, demonstrating their ability to model fluctuations in energy prices effectively. Similarly, [8] utilized ARIMA and GARCH models for traffic modeling and prediction in telecommunication networks, emphasizing the models’ robustness in handling complex network traffic data. Furthermore, [9] illustrated the integration of ARIMA and GARCH models for forecasting the USD/EUR exchange rate, highlighting the enhanced prediction accuracy achieved through this combination. The hybrid ARIMA-GARCH model is particularly effective in financial forecasting, where both linear patterns and volatility need to be accounted for. [10] demonstrated the superiority of this hybrid model in gold price forecasting, where it significantly improved forecasting accuracy by addressing both linear trends and volatility. Additionally, comparisons between traditional time series models and machine learning models have garnered attention in the

Sigma J Eng Nat Sci, Vol. 43, No. 1, pp. 199−212, February, 2025

literature. Studies by [11,12], and [13] have highlighted the potential of neural network models in capturing complex patterns in data, particularly in finance and economics. The Transformer model, a deep learning approach leveraging self-attention mechanisms, has gained prominence in time series analysis due to its ability to capture long-term dependencies. Unlike traditional models like ARIMA and GARCH, which rely on past values and variances, the Transformer model utilizes self-attention to weigh the importance of different time steps. This makes it particularly effective for complex and irregular time series data [14,15]. However, the Transformer model is not without limitations. One significant challenge is its computational inefficiency, especially concerning the self-attention mechanism, which scales quadratically with the length of the input sequence. This can become a bottleneck when dealing with long time series, leading to substantial time and memory complexity [3, 16]. Additionally, standard Transformers may struggle with capturing local dependencies in time series data, which is crucial for accurate forecasting and anomaly detection. Handling missing data is a critical aspect of time series analysis, as it directly impacts the accuracy and reliability of the results. Various imputation methods are employed to address this challenge, each with its strengths and limitations. Commonly used techniques include Last Observation Carried Forward (LOCF), Next Observation Carried Backward (NOCB), Mean Imputation, Linear Interpolation, Seasonal-Trend Decomposition using Regression (STR), Moving Average, Regression Imputation, and Kalman Filtering [17-20]. The choice of imputation method is critical in time series analysis, where preserving temporal dependencies and trends is essential. Research by [21,22] emphasizes that assumptions of linearity and stationarity in time series data may not always hold, making the selection of imputation methods even more crucial. More recent advancements in imputation methods are highlighted by [23], who introduced a modified genetic algorithm for the Travelling Salesman Problem, featuring novel crossover and mutation operators that could be adapted for time series imputation. Additionally, [24,25] presented numerical methods for solving complex differential equations, which could enhance imputation accuracy in datasets with unique structural characteristics. The dual hesitant fuzzy set theoretic approach in fuzzy reliability analysis, as discussed by [26,27], offers a theoretical foundation that could further improve the accuracy and robustness of imputation methods, particularly in systems characterized by uncertainty. In comparing the performance of ARIMA, GARCH, and Transformer models across different imputation scenarios, it is essential to consider how each method handles missing data. Studies by [28,29] have shown that Transformer models generally outperform recurrent neural networks (RNNs) across different imputation methods, particularly with Stineman interpolation. This suggests

Sigma J Eng Nat Sci, Vol. 43, No. 1, pp. 199−212, February, 2025

that while traditional models like ARIMA and GARCH are robust in many scenarios, Transformer models may offer superior performance in handling incomplete time series data, especially when advanced imputation techniques are applied. In conclusion, the effectiveness of imputation methods in time series analysis depends on the specific characteristics of the data and the research objectives. The methods reviewed in this section provide a comprehensive framework for understanding the efficacy of different imputation techniques in various scenarios, particularly when applied to traditional time series models and Transformer models. Although there is a wealth of research on time series forecasting and missing data imputation, several gaps remain. First, while traditional models like ARIMA and GARCH have been well-studied, their comparative performance against modern models like Transformers, especially in the context of missing data, is not fully understood. Additionally, the effectiveness of various imputation techniques when applied to these models has not been comprehensively evaluated, particularly across different levels of data completeness and sample sizes. This gap in the literature necessitates a thorough investigation into which imputation techniques best complement specific forecasting models under varying conditions. To address these gaps, this study systematically compares the performance of ARIMA, GARCH, and Transformer models in handling missing data across different scenarios. By evaluating eight different imputation methods—ranging from conventional techniques like mean imputation and linear interpolation to more advanced methods like Kalman filtering—this research aims to identify the most effective combinations of models and imputation techniques. The study employs a comprehensive simulation approach, analyzing different sample sizes and varying levels of missing data to provide a robust framework for selecting the most appropriate methods in time series forecasting. This study advances the existing literature by offering a detailed, comparative analysis of traditional and modern time series models in conjunction with a wide range of imputation methods. By integrating advanced techniques and considering various scenarios, this research not only fills critical gaps in the literature but also introduces a new framework for evaluating the effectiveness of different approaches to missing data in time series analysis. The findings have the potential to significantly improve the accuracy and reliability of time series forecasting, particularly in fields where data completeness is a challenge. For this purpose, the study examines eight imputation methods through a comprehensive simulation, considering three different sample sizes and varying imputation rates. This analysis provides valuable insights into possible combinations of estimation methods and imputation techniques. The inclusion of advanced imputation techniques, such as those discussed by [5], [30,31], and [27], enriches the discussion by integrating novel approaches. Furthermore, the

study compares different methods while also exploring the potential for transitioning from conventional to more sustainable methods, as discussed by [32], suggesting new avenues for enhancing imputation techniques using AI-based methods. In the rest of the study, ARIMA and GARCH, which are traditional methods frequently used in time series analysis, and Transformer models, which are innovative methods, are presented in the Method section. Additionally, information is provided about eight different imputation methods. The structural, metric, and scenario parameters of the simulation study and the information and steps for application are introduced in the Analysis section. In the Results section, all findings are presented in detail for all cases. In the Conclusion section, the findings are discussed.

Materials And Methods

This section outlines the methodologies employed in this study, including the models and imputation techniques used for analysis. Estimators The ARIMA model and the GARCH model (are popular methods for time series forecasting and volatility modeling. ARIMA is a widely used time series analysis model for forecasting and understanding time-dependent data. The ARIMA model is denoted as ARIMA (p, d, q), where p represents the autoregressive order, d represents the differencing order, and q represents the moving average order. The model’s equations involve the autoregressive terms, moving average terms, and the differencing operator, which are used to capture the temporal dependencies and trends in the data [33,34]. The ARIMA model equation can be represented in Eq.1: (1) where Yt is the value of the time series at time t. c is the constant term or intercept. ϕi (i = 1,2,…, p) are the autoregressive parameters representing the effect of past values on the current value. Yt-i are the lagged values of the time series. ϕj (i = 1,2,…, q) are the moving average parameters representing the effect of past errors on the current value. ϵt is the error term at time t, assumed to be white noise with mean zero and constant variance. The p and q parameters represent the order of the autoregressive and moving average components, respectively. The Integrated (l) component indicates the number of differences needed to make the time series stationary. GARCH is a model used to analyses and forecast the volatility of time series data. The GARCH model is denoted as GARCH (p, q), where p represents the order of the GARCH terms, and q represents the order of the ARCH terms. The

Sigma J Eng Nat Sci, Vol. 43, No. 1, pp. 199−212, February, 2025

model’s equations involve the conditional variance, which captures the time-varying volatility in the data [35]. The basic GARCH model equation can be represented in Eq.2:

(2) where σt2 is the conditional variance of the time series at time t. ω is the constant term or intercept of the GARCH model. α1​is the coefficient of the lagged squared error 2 term, representing the persistence of volatility shocks. ϵt-1 is the squared error term at time t - 1. β1 is the coefficient of the lagged conditional variance term, representing the 2 is the conditional varidecay of past volatility shocks. σt-1 ance at time t - 1. The Transformer model utilizes self-attention mechanisms to capture dependencies across the entire sequence [6]. The equations for the Transformer model are:

where Z(l) represents the output of the l -th layer, and LayerNorm is a normalization layer, MultiHeadAttention is the multi-head attention mechanism, FeedForward is a feedforward neural network layer. These equations highlight the fundamental elements of the Transformer model: the multi-head attention mechanism, which enables the model to attend to various segments of the input sequence, and the feedforward neural network, which processes the aggregated attention data. Layer Normalization is employed to normalize the inputs of each layer, aiding in the stabilization and acceleration of deep neural network training. The Transformer model is a type of neural network architecture that has been applied to time series analysis. It utilizes self-attention mechanisms to capture dependencies between different time steps in the data. The equations of the Transformer model involve the self-attention mechanism, which allows the model to weigh the importance of different time steps when making predictions [6]. An adaptation of the Transformer model equation for time series analysis includes the following components: input representation, positional encoding, transformer encoder, transformer decoder, and output layer. Scaled dot-product Attention: (4) In the equation above: Q represents the query. K represents the key. V represents the value in the attention mechanism. dk is the dimension of the keys (KT: The transpose of the key matrix K).

In the equations above: PE(pos,2i) and PE(pos,2i+1) represent the positional encoding for even and odd indices, respectively. pos represents the position. dmodel is the dimension of the model. Imputation Techniques The imputation techniques utilized in the comparison study include: a. Last Observation Carried Forward (LOCF): Fills missing values with the last observed value [36]. b. Next Observation Carried Backward (NOCB): Fills missing values with the next observed value [37]. c. Mean Imputation: Replaces missing values with the mean of the available data [37]. d. Linear Interpolation: Estimates missing values based on linear interpolation between adjacent data points [38]. e. Seasonal Decomposition: Decomposes the time series into seasonal and trend components, filling missing values based on the decomposition [39]. f. Moving Average: Fills missing values with the average of neighboring data points within a specified window [40]. g. Regression Imputation: Predicts missing values using regression analysis based on available data [41]. h. Kalman Filtering: Utilizes Kalman filter algorithms to estimate missing values based on observed data and system dynamics [42]. These techniques offer diverse approaches to handling missing data in time series analysis, each with its strengths and limitations.

Analysis

Simulation An inclusive simulation study was conducted for the analysis of the study. Simulation inputs are presented in detail in Table 1. This detailed procedure outlines how to conduct the comparison of the specified models using various imputation methods on different types of data and sample sizes [43]. The RMSE (Root Mean Square Error) metric is used to evaluate the accuracy of each model under different conditions. RMSE is related to bias and variance in the context of model evaluation and is therefore considered the metric used in this study. With the abundance of scenarios and tables, all results are presented with RMSE for interpretability. The RMSE is represented as:

Sigma J Eng Nat Sci, Vol. 43, No. 1, pp. 199−212, February, 2025

where yi is the actual value. yî is the predicted value. n is the number of observations. The simulation procedure systematically evaluates the performance of eight imputation techniques and various estimation models across different scenarios, including the Transformer, ARIMA, and GARCH models. The analysis stages are as follows: - A sample time series dataset is generated for each combination of estimation model, imputation method, and sample sizes, with missing values introduced based on specified percentages and imputation methods. - The missing values are then imputed using the chosen method, and each model is trained and evaluated using the imputed data. - RMSE values are calculated for each model, with the process repeated for 1000 iterations to capture variability.

The RMSE values are aggregated over the iterations to derive average RMSE values for each combination. - Finally, comparisons are made across different models, imputation methods, data cases, and sample sizes to determine their relative performance. All applications in this study were conducted in R using RStudio IDE. Steps of simulation is also shared in Table 2. This table outlines the simulation process, detailing each step and the corresponding operation performed. Tables 3-7 are derived from the results of the simulation study and are discussed in detail in the Results section. Application In addition to the simulation study, an application based on real data was conducted to evaluate the practical performance of different imputation methods and time series models. For this application, historical stock data for AAPL (Apple Inc.) was utilized. The analysis utilized data consisting of 756 observations, covering approximately three years of time series data. Thus, a supportive approach was adopted in the simulation studies with sample sizes of 200, 600, and 1000.

Methods

ARIMA (AutoRegressive Next Observation Moving Average Integrated Moving Average) Carried Backward Model (NOCB)

GARCH (Generalized Autoregressive Conditional Heteroskedasticity) Model

Iterations, repetitions, sample widths, imputation percentages, methods, model types

`aggregate(RMSE ~ Sample_Width + Imputation_Percentage + Imputation_Method + Model_Type, data = results_df, FUN = mean)`

Sigma J Eng Nat Sci, Vol. 43, No. 1, pp. 199−212, February, 2025

The analysis began with loading essential R libraries. Historical adjusted closing prices were obtained. To simulate missing data based on randomness, function based on uniform distribution was applied, creating data gaps at levels of 0%, 10%, 25%, and 40%. Eight imputation techniques were employed, such as LOCF, NOCB, mean imputation, linear interpolation, seasonal decomposition, moving average, regression imputation, and Kalman filtering. The data was then used to fit three models: The Transformer model, ARIMA, and GARCH. The performance of each model was assessed by calculating the RMSE for every combination of imputation method and missing data level. Results were systematically compiled into a table, presented as Table 8, and the findings are discussed in the results section. The application provides insight into the extent to which the simulation results align with those obtained from real-world

data analysis and highlights the implications of the study’s findings for practical applications.

Results And Discussion

To analyses the results, we can examine the RMSE values for each combination of model, imputation method, and missing data percentage. Table 3 shows the analysis results for the case where the number of samples is 200. In terms of estimators, across all missing data percentages (0%, 10%, 25%, 40%), the Transformer model generally performs better than ARIMA and GARCH models in terms of RMSE. This suggests that the Transformer model is more effective in predicting time series data compared to traditional ARIMA and GARCH models. Regarding imputation methods, among the imputation methods, the performance varies depending on the

TraMod ARIMA GARCH TraMod ARIMA GARCH TraMod ARIMA GARCH TraMod ARIMA GARCH LOCF

* LOCF: Last Observation Carried Forward, NOCB: Next Observation Carried Backward, Mean: Mean Imputation, Linear: Linear Interpolation, Seasonal: Seasonal Decomposition, MovAve: Moving Average Imputation, Regression: Regression Imputation, KalmFil: Kalman Filter Imputation; TraMod: Transformer Model, ARIMA: AutoRegressive Integrated Moving Average, GARCH: Generalized AutoRegressive Conditional Heteroskedasticity.

TraMod ARIMA GARCH TraMod ARIMA GARCH TraMod ARIMA GARCH TraMod ARIMA GARCH LOCF

* LOCF: Last Observation Carried Forward, NOCB: Next Observation Carried Backward, Mean: Mean Imputation, Linear: Linear Interpolation, Seasonal: Seasonal Decomposition, MovAve: Moving Average Imputation, Regression: Regression Imputation, KalmFil: Kalman Filter Imputation; TraMod: Transformer Model, ARIMA: AutoRegressive Integrated Moving Average, GARCH: Generalized AutoRegressive Conditional Heteroskedasticity.

Sigma J Eng Nat Sci, Vol. 43, No. 1, pp. 199−212, February, 2025

TraMod ARIMA GARCH TraMod ARIMA GARCH TraMod ARIMA GARCH TraMod ARIMA GARCH LOCF

* LOCF: Last Observation Carried Forward, NOCB: Next Observation Carried Backward, Mean: Mean Imputation, Linear: Linear Interpolation, Seasonal: Seasonal Decomposition, MovAve: Moving Average Imputation, Regression: Regression Imputation, KalmFil: Kalman Filter Imputation; TraMod: Transformer Model, ARIMA: AutoRegressive Integrated Moving Average, GARCH: Generalized AutoRegressive Conditional Heteroskedasticity.

combination of model and missing data percentage. For example, the Kalman filter, mean, and regression imputations tend to perform relatively well across different models and missing data percentages. However, the effectiveness of imputation methods can vary based on the specific characteristics of the data and the modeling approach. Table 4 shows the analysis results for the case where the number of samples is 600. Similar to the previous analysis, the Transformer model generally outperforms ARIMA and GARCH models across different missing data percentages (0%, 10%, 25%, 40%). This consistency suggests that the superiority of the Transformer model in predicting time series data is robust and not heavily influenced by missing data. The performance of imputation methods varies across different models and missing data percentages. For instance, Kalman filter, mean imputation, and NOCB imputations show relatively stable performance across various scenarios, indicating their effectiveness in handling missing data in time series analysis. However, some methods like LOCF, linear interpolation, seasonal decomposition, moving average exhibit fluctuating performance depending on the combination of model and missing data percentage. As observed in the previous analysis, higher percentages of missing data lead to higher RMSE values across all models and imputation methods. This consistent trend emphasizes the detrimental effect of missing data on the accuracy of time series predictions. Table 5 shows the analysis results for the case where the number of samples is 1000. The RMSE values for the Transformer model are consistently lower compared to ARIMA and GARCH models, indicating better predictive performance. For this scenario, some imputation methods, such as mean, and regression imputations show relatively stable performance across different scenarios. As observed in previous analyses, higher percentages of missing data lead to higher RMSE values across all models and imputation methods. This trend underscores the importance of

handling missing data effectively in time series analysis to maintain prediction accuracy. To examine the estimation model, Table 6 provides useful summary information. The Transformer model has been combined to present the lowest RMSE value 5 times with regression imputation, 3 times with NOCB imputation and 1 time with Kalman filter. Therefore, regression imputation and NOCB imputation are compatible with the Transformer model. The ARIMA model has been combined to present the lowest RMSE value 7 times with mean imputation, 2 times with Kalman filter. Therefore, mean imputation and Kalman filter are compatible with the ARIMA model. The GARCH model has been combined to present the lowest RMSE value 5 times with Kalman filter imputation, 3 times with mean imputation and 1 time with linear interpolation. Therefore, Kalman filter and mean imputation are compatible with the ARIMA model. To examine sample sizes, Table 7 provides useful summary information. LOCF was not applied to any of the datasets. NOCB was applied 2 times to the dataset with size 600 and once to the dataset with size 1000 but was not applied to the dataset with size 200. Mean imputation was applied 3 times to the dataset with size 200, 3 times to the dataset with size 600, and 4 times to the dataset with size 1000, indicating consistent usage across all dataset sizes. Linear interpolation was applied once to the dataset with size 1000 and not to the datasets with sizes 200 or 600. Seasonal decomposition and moving average were not applied to any of the datasets. Regression imputation was applied 2 times to the dataset with size 200, once to the dataset with size 600, and 2 times to the dataset with size 1000, showing moderate usage. Kalman filter was applied 4 times to the dataset with size 200, 3 times to the dataset with size 600, and once to the dataset with size 1000, indicating frequent usage, especially for smaller datasets. In general, mean and Kalman filter imputation methods had high usage, NOCB and regression imputation had moderate usage, linear interpolation had

Sigma J Eng Nat Sci, Vol. 43, No. 1, pp. 199−212, February, 2025

Table 6. Lowest RMSE (Best Combination) Counts according to the Estimation Methods TraMod

Garch

Kalman Filter has 1 count for TraMod, 2 for ARIMA, and 5 for GARCH

* LOCF: Last Observation Carried Forward, NOCB: Next Observation Carried Backward, Mean: Mean Imputation, Linear: Linear Interpolation, Seasonal: Seasonal Decomposition, MovAve: Moving Average Imputation, Regression: Regression Imputation, KalmFil: Kalman Filter Imputation; TraMod: Transformer Model, ARIMA: AutoRegressive Integrated Moving Average, GARCH: Generalized AutoRegressive Conditional Heteroskedasticity.

Table 7. Lowest RMSE (Best Combination) Counts according to the Sampe Sizes 200

* LOCF: Last Observation Carried Forward, NOCB: Next Observation Carried Backward, Mean: Mean Imputation, Linear: Linear Interpolation, Seasonal: Seasonal Decomposition, MovAve: Moving Average Imputation, Regression: Regression Imputation, KalmFil: Kalman Filter Imputation; 200, 600, and 1000 represent sample sizes.

low usage, and LOCF, seasonal decomposition, and moving average were not used. These observations can help in determining the most suitable imputation methods based on dataset size and missing data patterns. The frequency values ​​in Table 6 and Table 7 are presented comparatively in Figure 1. The combination of two plots in Figure 1 offers a comprehensive overview of how imputation methods are distributed across both model types and dataset sizes. The consistent use of visual elements such as transparency for zero counts, color differentiation, and well-placed labels ensures clarity and readability, facilitating quick comparison across different categories. For instance, LOCF, Moving Average, and Seasonal Decomposition are absent in both figures, confirming they were not used for any dataset. On the other hand, it is seen that Kalman filtering and Mean methods dominate the process. Considering the values of RMSE values in the table and the fact that there are many scenarios and combinations, graphs of imputation techniques and estimation methods

for sample sizes of 200, 600 and 1000 are presented in Figures 2-4 for clarity. The Transformer model is superior to the ARIMA and GARCH models in complete data (original data) and missing data completion scenarios at all imputation rates. Moreover, the transformer model was able to provide a lower RMSE value than the original data in case of 10% imputation. This indicates the success of imputation techniques in completing missing data at low imputation rates. Although the GARCH model had superior results compared to the ARIMA model in the absence of imputation, in the case of imputation, the ARIMA model performed better than the GARCH model for all imputation rates. However, contrary to this general trend, the GARCH model outperformed ARIMA at all imputation rates when imputed with Kalman Filtering under the n = 600 scenario. The GARCH model again performed better than ARIMA when imputed with NOCB, Mean, Linear, Seasonal Decomposition, and Kalman filter imputation methods under the n = 1000 scenario.

Sigma J Eng Nat Sci, Vol. 43, No. 1, pp. 199−212, February, 2025

Sigma J Eng Nat Sci, Vol. 43, No. 1, pp. 199−212, February, 2025

Sigma J Eng Nat Sci, Vol. 43, No. 1, pp. 199−212, February, 2025

TraMod ARIMA GARCH TraMod ARIMA GARCH TraMod ARIMA GARCH TraMod ARIMA GARCH LOCF

* LOCF: Last Observation Carried Forward, NOCB: Next Observation Carried Backward, Mean: Mean Imputation, Linear: Linear Interpolation, Seasonal: Seasonal Decomposition, MovAve: Moving Average Imputation, Regression: Regression Imputation, KalmFil: Kalman Filter Imputation; TraMod: Transformer Model, ARIMA: AutoRegressive Integrated Moving Average, GARCH: Generalized AutoRegressive Conditional Heteroskedasticity

In the application, the Transformer model consistently outperforms the ARIMA and GARCH models across different imputation methods and missing data levels, highlighting its effectiveness in dealing with incomplete data. For low levels of missing data (10%), mean imputation performs best with both the Transformer model and ARIMA, suggesting that conventional imputation methods are effective when data completeness is relatively high. However, as the level of missing data increases to medium (25%) and high (40%) levels, Kalman Filtering emerges as the superior imputation method, providing the most accurate results across all models. Specifically, at high levels of missing data, Kalman Filtering shows the best performance with the GARCH model. On the other hand, for lower and medium missing data levels, NOCB is particularly effective with the GARCH model, outperforming other imputation methods. The results in Table 8 and their corresponding analyses are more easily interpreted with the graphs in Figure 5. Figure 5 presents RMSE results for the estimation models and imputation techniques across different levels of missing data in the AAPL dataset. The missing data levels are represented as percentages (0%, 10%, 25%, 40%). The RMSE values are shown for three forecasting models: the Transformer model, ARIMA, and GARCH. These models are assessed using eight imputation methods: LOCF, NOCB, Mean, Linear Interpolation, Seasonal Decomposition, Moving Average, Regression, and Kalman Filtering. In the case of imputing 10% and 25% missing data, the Transformer model and ARIMA produce close and consistent RMSE values. The differences between imputation methods become more pronounced with higher missing data levels. However, this pattern exhibits more volatility for the GARCH model. Additionally, the results from the Transformer model and ARIMA show close alignment. Moreover, as the level of missing data increases (from 10% to 40%), there is a noticeable increase in RMSE across all models and imputation methods. This trend is consistent, demonstrating that higher missing data levels generally degrade model performance.

Conclusion

The results show that the choice of imputation technique significantly influences the accuracy of predictions in time series analysis. Techniques such as mean imputation and Kalman filter imputation have shown reliability and effectiveness across different model types, while methods like LOCF, seasonal decomposition, and moving average were less utilized, potentially due to their unsuitability for the given data. The Transformer model shows promise as a predictive modeling technique, consistently outperforming traditional ARIMA and GARCH models in various scenarios. The Transformer’s self-attention mechanism allows it to capture long-term dependencies more effectively than ARIMA and GARCH models. Results emphasize the

Sigma J Eng Nat Sci, Vol. 43, No. 1, pp. 199−212, February, 2025

importance of careful consideration when dealing with missing data and selecting appropriate imputation methods to enhance the accuracy of time series analysis. Higher percentages of missing data imputation generally lead to higher RMSE values across all models and imputation methods. This indicates that imputation rate can significantly affect the accuracy of time series predictions, regardless of the modeling approach used. The decrease in RMSE values as the number of samples increases is an important indicator of the consistency of the analyses, aligning theoretical expectations with empirical observations. Specifically, at low levels of missing data (10%), mean imputation is highly effective with both the Transformer model and ARIMA, suggesting that traditional imputation methods work well when data is mostly complete. However, as the proportion of missing data increases (25% and 40%), Kalman filtering becomes the superior method, yielding the most accurate predictions across all models. The GARCH model shows more volatility in performance compared to Transformer and ARIMA, with Kalman filtering particularly beneficial at high missing data levels. Interestingly, NOCB imputation performs well with the GARCH model at lower and medium missing data levels. Despite the varied application of techniques, this study reflects an exploratory approach to determine the best imputation method for different model types and datasets. These insights can guide future imputation strategy choices, emphasizing techniques that proved useful and exploring underutilized methods to potentially enhance model performance. The Transformer model’s superiority is evident across all scenarios, and Kalman filtering emerges as the most reliable imputation method as missing data levels increase. One potential limitation of this study is the computational complexity and scalability of the Transformer model. Transformer’s self-attention mechanism, while powerful, can be computationally intensive, making it less feasible for very large datasets or in resource-constrained environments. Future research should identify challenges with time counters and real root values; ​​and investigate optimizations and alternatives for the solution. Additionally, it would be beneficial to further explore and evaluate the performance of underutilized imputation methods, such as seasonal decomposition and moving average, to determine their potential effectiveness in different contexts. Investigating the impact of varying sample sizes on the robustness of imputation methods and predictive models can also provide deeper insights.

Acknowledgements

The previous abstract version of the study was presented at the “9th International Conference on Business Management and Economics” as an oral presentation with the title “Imputation Methods Effect on the Goodness of Fit of the Statistical Model” [43]. This study was supported by

Sigma J Eng Nat Sci, Vol. 43, No. 1, pp. 199−212, February, 2025

Eskişehir Technical University Scientific Research Project Commission under grant no: 24ADP185.

Data Availability Statement

Raw data that support the finding of this study are available from the corresponding author, upon reasonable request.

Conflict Of Interest

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics

There are no ethical issues with the publication of this manuscript.

Share and Cite