Time-varying parameter estimation and adaptive Kalman filter in computer aided control application

Abstract

Keywords: System; Identification; Parameter estimation; Adaptive Kalman filter; Control

Introduction

Nowadays, different methods are used for system identification and control. Considering how the system responds to given commands, system model may be developed, and system control may be performed over this model in the manner that it will adapt to changes in environmental conditions. Such kind of methods are classified as adaptive control methods. On one hand, these methods called “the best control methods” focus on getting the system to desired target values; on the other hand, they minimize an identified cost function.

This study deals with the development of the system model considering the system’s response to given inputs and the system control method on the developed model. In the first section, ARX (autoregressive exogenous input) models which are frequently used in system identification have been introduced and explanations about related problems have been made. In the second section, information on Weighted Least Squares method which is used for estimating the parameters in ARX models have been given. After making explanations about the studies on encountered problems and their solution methods, Kalman Filter

*Corresponding author. *E-mail address: ozbek@science.ankara.edu.tr This paper was recommended for publication in revised form by Regional Editor Ali Hakan Büyüklü Published by Yıldız Technical University Press, İstanbul, Turkey Copyright 2021, Yıldız Technical University. This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Sigma J Eng Nat Sci, Vol. 39, No. 4, pp. 343-350, December, 2021

which is another frequently used application and its solution methods have been introduced as well. In the third section, “the least variance” control rule has been explained briefly. As it is known, success in employing this method depends on the success of parameter estimations existing in the ARX model. In the fourth section, simulation study has been made on how the methods of Successive Weighted Least Squares, Normal Kalman Filter and Corrected Kalman Filter work in the case that they are used for the identification process of a system which has time-varying parameters. In the fifth section, an experimental study has been made in order to make temperature control using the Corrected Kalman Filter which was explained in the second section and the Least Variance control rule which was explained in the third section.

System Identification

system are the most important two problems in system identification. Several methods for the estimation of the unknown parameter vector and related encountered problems have been discussed broadly in [1–16]. Especially in cases where the system’s parameters are time-variable, it is observed that estimations obtained using estimator have deviated from the actual values, and therefore that the estimator must be corrected to some extent. Several methods are used for the estimation of model parameters. Only the Least Squares (LS) method and Kalman Filter have been emphasized in this paper.

Recursive Weighted Least Squares

METHOD (RWLS) For the purpose of estimating the parameters in the model given in Equation (3), if the cost function 2

1 This section deals with the methods on estimation= of VN (θ ) β (N , t )  y(t ) − θ T φ (t ) t = 1, 2, ..., N (4) ∑ parameters in a model in the case that a system is modelled N t =1 with ARX and addresses to several problems. Let a system is minimized with respect to θ, LS estimator of θ is with input (control variable) {u(t)}, and output {y(t)} is found as modelled with linear difference equation N

y(t) + a1y(t – 1) + ... + any(t – n) = b1u(t – 1) + ... + bmu(t – m)+ ν(t)

N  N  θˆ(N ) =  ∑ β (N , t )φ (t )φ T (t )  ∑ β (N , t )φ (t ) y(t ) (5) (1) =  t 1=  t 1 

here, {ν(t)} shows white noise process and t = Under the assumption that new data will include more information compared to the older, it may be selected as 1, 2, ..., N shows time periods. Equation (1) may be –1 –1 written using delay operator q ; q y(t) = y(t – 1) β(t,k) = λ(t)β(t – 1, k), 1 ≤ k ≤ t – 1

B(q–1) = b1q–1 + b2q–2 + ... + bmq–m and n, m are lag of the model; a1, ..., an, b1, ..., bm are unknown parameters of the model. Models (1) or (2) show the dynamic relation between input and output, and they are known as ARX models in the literature. Given that θT = (a1, ..., an, b1, ..., bm) φT(t) = (–y(t – 1)), ..., –y(t – n)u(t –1), ..., u(t – m)) the models (1) or (2) may be written as y(t) = θTφ(t) + ν(t)

Estimating the unknown parameter vector and determining the lag of the model which best represents the

may be taken instead of Equation (6). If it is considered that λ(k) ≤ 1 and λ(k) = λ for every k, result β(t,k) = λt–k

is obtained from Equation (7). If Equation (8) is used in cost function (4), it is assumed that the effects of new data will be more powerful, and the effects of old data will be weaker in the cost function. In other words, old data are being forgotten. Therefore, λ is called “forgetting factor”. Estimator given with (5) is not recursive. In order to get the recursive estimator, if taken as t

Sigma J Eng Nat Sci, Vol. 39, No. 4, pp. 343-350, December, 2021

is obtained. Using the equalities (9) and (10) in Equation (5), and considering that P(t) = R–1(t), RWLS algorithm is given as

P(t − 1)φ (t ) = P(t )φ (t ) λ (t ) + φ T (t )P(t − 1)φ (t )

1  P(t − 1)φ (t )φ T (t )P(t − 1)   P(t − 1) − λ (t )  λ (t ) + φ T (t )P(t − 1)φ (t ) 

matrix is unitary. The filtering problem is the problem of determining the best estimate of its θ(t) condition, given its observations y(t), y(t – 1), ..., y(1) (Kalman, [17]. When y(t), y(t – 1), ..., y(1) observations are given, the estimation of state θ(t) with θˆ(t t − 1) and the covariance matrix of the error with P(t) when y(t), y(t – 1), ..., y(1) observations are given, the estimation of state θ(t) with θˆ(t t − 1) and the covariance matrix of the error are shown with P(t + 1|t). Let the initial state be assumed to have a normal distribution. The optimum update equations for Kalman Filter (KF) are

Estimation parameters (11)-(13) are commonly used for systems which have time-varying parameters. As seen in the algorithm, the algorithm gain reduces when matrix P shrinks, and thus the estimation deviates from the actual value. Forgetting factor λ is used to neutralize the effect of this problem. Many amplified RWLS algorithms have been proposed for the problem of the time-varying parameter estimation. In cases where the parameters given by Equation (3) are time-varying, various corrections in LS method have been made in order to estimate these parameters. Especially these methods have focused on selecting the forgetting factor and correcting the P matrix. Different perspectives have been used for successive calculation of the forgetting factor and correction of P matrix [4-13]. Selection of Forgetting Factor Forgetting factor λ is selected as a value close to 1 and taken as λ = 0.90, λ = 0.95 or λ = 0.99 [1, 2]. In general. Additionally, given that Σ0 is a small constant (for example, Σ0 = 0.05), it may be also selected as 1 − φ T (t )L(t ) ε (t )2 λ (t )= 1 − Σ0

Estimation of Parameter Vector with Kalman Filter One of the methods used for estimating the parameters in the model given in Equation (3) is Kalman Filter. If the parameter vector is regarded as random walk progress for this purpose, Equation (3) may be written as state space model θ(t + 1) = θ(t) + w(t)

θˆ(t t=) θˆ(t t − 1) + K (t )  y(t ) − φ T (t )θˆ(t t − 1) (18) K (t ) =P(t t − 1)φ (t ) φ T (t )P(t t − 1)φ (t ) + R2 (t ) P(t + 1| t ) = P(t ) + R1 (t )  I − K (t )φ T (t ) P(t t − 1) . P(t ) =

In the above equations θˆ(t t − 1) is the a priori estimation and θˆ(t t ) is the a posteriori estimation of θ(t). Also, P(t + 1|t) and P(t) are the covariance of a priori and a posteriori estimations respectively. Hagglund [11] has proposed a test which detects the change of parameters of model (4.3) and made corrections on Kalman Filter accordingly. Beltran [13] has proposed a test which detects the change in parameter for a particular situation (for the case that there is no system input) of model (3) and made corrections on the filter accordingly. Filter correction processes in the case that the model is constructed incorrectly are discussed in [4-13]. Considering that the filter is used online, it becomes more of an issue to perform the correction processes with very few calculations.

Feedback Control

Feedback input (control) ut at time t, depending on values {y(0), ..., y(t), u(0), ..., u(t – 1)}, is generated by minimizing function 2 1 t = J t lim ∑  y(i) − y * (i ) t →∞ t i =1

Here, state vector is the parameter vectors {W(t)} and 1  {e(t)} is the white noise processes; E[w(t)wT(s)] = R1(t) = ut aˆ1,t yt + ... + aˆn,t yt −n +1 − bˆ2,t ut −1 − ... − bˆm,t ut −m +1 + y*t +1   δts, E[e(t)e(s)] = R2(t)δts, E[w(t)e(s)] = 0, E[θ(0)] = θ̄(0), bˆ1,t  T E[(θ0 – θ̄0) (θ0 – θ̄0) ] = P(0) and θ(0) have been assumed (23) as independent from w(t) and ν(t). System’s transition

Sigma J Eng Nat Sci, Vol. 39, No. 4, pp. 343-350, December, 2021

This method is known as self-adaptive least variance feedback control method. Here, {yi*} shows the targeted value (reference value) of the system output during time t and aˆ1,t ,...aˆn,t , bˆ1,t ,..., bˆm,t the estimations of the parameters existing in the system during time t [1, 2, 3, 16]. As it is understood from this control rule, keeping the system at the desired target value depends on correct estimation of the parameters.

Simulation Study

This section deals with model (2) and includes a simulation study on parameter estimation using the different values of forgetting factor with the purpose of observing how RWLS method works in the case when the parameter is time-varying. Additionally, for this model, another simulation study has been made in order to compare Normal Kalman Filter (NKF) given in equalities (17) and (21) with Corrected Kalman Filter (CKF) proposed by Özbek [18, 20]. Let the model for the simulation study be as follows: y(t) = a1(t)y(t – 1) + ν(t)

y(0) 10,= θˆ(0) 0,= P(0) 1 is Here, ν(t) ~ N(0,1), = regarded as −0.9, 1 ≤ t ≤ 100  = a1 (t ) 0.9, 101 ≤ t ≤ 200 .  −0.9, 201 ≤ t ≤ 400  In RWLS method given with equalities (11) and (13), given that λ = 1, the astimarted values obtained from the algorithm, matrix P and the residues are given in Figure 1, Figure 2 and Figure 3. As seen in Figure 1, estimatedvalues of the parameter in the case that λ = 1 deviate from the actual value. In other words, the algorithm could not track this change. As it is seen in Figure 2, P matrix moves to zero, therefore the gain obtained from the algorithm decreases. Given that λ = .99, matrix P and residues are shown in Figure 4, Figure 5 and Figure 6. Comparing Figure 4 with Figure 1, a small improvement is seen in the estimation. Again, when Figure 5 and Figure 2 are compared, it is seen that matrix P did not move but resulted as very close to zero.

Given that λ = .9, estimatedvalues obtained from the algorithm, matrix P and residues are given in Figure 7, Figure 8 and Figure 9. As is seen in Figure 7, the parameter estimations are improving significantly. As is seen in Figure 8, matrix P resulted as greater than other cases.

Sigma J Eng Nat Sci, Vol. 39, No. 4, pp. 343-350, December, 2021

Given that λ = .8, estimatedvalues obtained from the algorithm, matrix P and residues are given in Figure 10, Figure 11 and Figure 12. As is seen in Figure 10, the algorithm may track the changes but oscillates greatly around the actual value. As is seen in Figure 11, matrix P has greatened significantly. The difference between CKF and NKF as given by Özbek [18, 20] is merely in Equation (21). Instead of Equation (21), P*(t) = λ*(t)P(t)

is considered. Here, given that n indicates state vector and r indicates observation vector, T λ * (t=)  θˆ(t t ) − θˆ(t t − 1) P −1 (t t − 1) θˆ(t t ) − θˆ(t t − 1) +

T  y(t ) − φ T (t )θˆ(t t ) R2−1 (t )  y(t ) − φ T (t )θˆ(t t )  / (n + r )    

is the result. If λ*(t) ≤ 1, λ*(t) = 1 will be considered. Estimation values, P matrix and residues which have been obtained in order to compare NKF with CKF on the model given in Equation (24) are given in Figure 13 and Figure 18. When Figure 13 and Figure 14 are compared, it is seen that

Figure 14. Parameter estimation with NKF. CKF gives results more accurately than NKF does. When Figure 15 and Figure 16 are compared, it is seen that P in CKF does not move to zero. When Figure 17 and Figure 18 are compared, it is seen that residues in NKF are greater than those in CKF.

Sigma J Eng Nat Sci, Vol. 39, No. 4, pp. 343-350, December, 2021

in the model are assumed as constant in standard LS method. Nevertheless, in the case where the input-output model in Equation (3) is considered as state space model in equalities (15) and (16), it is assumed that the parameters have not constant but more of a probabilistic progress feature. This is one of the purposes for which Kalman Filter is used in this model. As is known, Kalman Filter could be obtained in accordance with different optimization measures, and least squares method is one of them. Detailed information on obtaining the Kalman Filter is given in [20-30].

Control With Adative Kalman Filter

Figure 18. Residues obtained with CKF. In RWLS method, forgetting factor must be selected before the algorithm starts working. As it is also understood from the simulation study, different selections of the forgetting factor cause the algorithm to give different results and it is not possible to foresee which one will suit better. Weighting the covariance matrix in CKF is arranged by the algorithm itself; there is no need to make a selection priorly. Since the weighting process requires less operations compared to other methods, CKF is more preferable than other methods for this model. As a result of changing the covariance matrix of white noise progress in Equation (15) mentioned in Section 2.2. and performing several simple operations, RWLS method given in equalities (11) and (13) could be obtained from the Kalman Filter given in equalities (18) and (21). In other words, RWLS method could be regarded as a special form of Kalman Filter. The parameters

In this section, an experimental study has been made in order to make temperature control using the Corrected Kalman Filter which was explained in the second section and the least variance control rule which was explained in the third section. 8-litre water initially at 25°C in an aquarium has been placed on a container filled with ice. The purpose was to keep the water’s temperature at 37.5°C. With such a purpose, several parts were included in the control system forming the testing apparatus. There were an A/D converter and a temperature sensor (LM 335 temperature sensor) for reading the water’s temperature and transmitting it into computer digitally. There were a 1500-Watt heater for increasing the water’s temperature, a D/A converter for sending the desired signal to the heater, and a triac module for setting the voltage coming from the computer between 0 and 220 Volts. There was also a mixer to facilitate the heat dissipation in the container filled with water. The system has been controlled online in accordance with the least variance feedback control rule. Under the assumption that the system is modelled with Equation y(t) = a(t)y(t – 1) + u(t – 1)+ ν(t)

CKF adjusted with equalities (25) and (26) has been used in order to estimate the parameter a(t), and the control signal has been calculated using Equation (23). Sampling interval has been taken as 1 second. Experimental system is shown in Figure 19. The output and input values obtained in accordance with the least variance feedback control rule are given in Figure 20 and Figure 21. Parameter estimation is shown in Figure 22. As is seen in Figure 20, the system’s output oscillates slightly around the target value. It is seen that the system is kept under control. In order to reduce this oscillation to a lower level, the control could be performed using the model which takes into consideration different delay times of output and input values. With the purpose of providing simplicity, Equation (27) has been regarded as the model of the system here.

Sigma J Eng Nat Sci, Vol. 39, No. 4, pp. 343-350, December, 2021

be seen from the simulation studies Corrected (Adaptive) Kalman Filter gives results more accurately than Normal Kalman Filter (NKF) for time varying parameter estimation. Corrected (Adaptive) Kalman Filter ensures that the system is kept under control by correctly estimating the parameter that changes over time. The use of the Corrected Kalman filter is recommended for similar control applications.

Acknowledgments

This study was carried out without any support from any institution, organization and person. Figure 19. Experimental system of temperature control.

Data Availability Statement

The authors confirm that the data that supports the findings of this study are available within the article. Raw data that support the finding of this study are available from the corresponding author, upon reasonable request.

Conflict Of Interest

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics

There are no ethical issues with the publication of this manuscript.

References

1. Ljung L, Söderström T. Theory and Practice of Figure 21. Input values of the system. Recursive Identification. Cambridge: The MIT Press; 1983.
2. Goodwin GC, Sin KS. Adaptive Filtering Estimation and Control. New Jersey: Prentice Hall; 1985.
3. Ljung L, Gunnarsson S. Adaptation and track- ing in system identification-a survey. Automatica 1990;26:7–21. [CrossRef]
4. Kulhavy R, Zarrop MB. On a general concept of for- getting. Int J Control 1993;58:905–24. [CrossRef] Simülasyon. İstanbul: Pigeon Yayıncılık; 2015.
5. Gunnarsson S. Combining and Regularization in (Turkish) Recursive Least Squares Identification. Department [22] Aliev FA, Ozbek L. Evaluation of convergence rate in of Electrical Engineering, Linköping University, the central limit theorem for the kalman filter. IEEE Linköping, Sweden, Technical Report No: 1871, Transactions On Automatic Control 1999;44:1905– 1987. 9. [CrossRef]
6. Guo L. Convergence and Logarithm Laws of Self- tuning Regulators. Automatica 1995;31:435–50. chastic stability of the adaptive fading extended [CrossRef] Kalman filter with the matrix forgetting factor.
7. Hagglund T. New estimation techniques for adap- Open Math 2016;14:934–45. [CrossRef] tive control. Ph.D. Thesis. Lund University, Lund, [24] Biçer C, Babacan EK, Özbek L. Stability of the Sweden, 1983. adaptive fading extended Kalman filter with the
8. Weston PF, Norton JP. Detection and Estimation matrix forgetting factor. Turk J Elec Eng Comp Sci of Abrupt Changes in Input State. Int J Control 2012;20:819–33. 1997;67:699–711. [CrossRef] [25] Özbek L, Efe M, Babacan EK, Yazıhan N. Online esti-
9. Beltran NDR. Autoregressive and adaptive estima- mation of capillary permeability and contrast agent tion with an application to hurricane track estima- concentration in rat tumors. Hacettepe Journal of tion. Int J Control 1995;26:1791–812. [CrossRef] Mathematics and Statistics 2010;39:283–93.
10. Xia Q, Rao M, Ying, Shen X. Adaptive fading kalman filter with an applications. Automatica vring target tracking. Journal of the Turkish 1994;30:1333–8. [CrossRef] Statistical Assocation 1999;2:193–206. [CrossRef]
11. Aström KJ, Wittenmark B. On self-tuning regula- tors. Automatica 1973;9:185–99. [CrossRef] of the discrete-time constrained extended kalman
12. Kumar PR, Varaiya P. Stochastic Systems: Estimation, fitler. Turkish Journal of Electrical Engineering & and Adaptive Control. New Jersey: Prentice Hall; Computer Sciences 2010;18:211–24. 1986. [28] Babacan EK, Ozbek L Efe M. Stability of the extended
13. Kalman RE. A new approach to linear filtering and kalman filter when the states are constrained. IEEE prediction problems. Journal of Basic Engineering Transactions on Automatic Control 2008;53:2707– 1960;82:35–45. [CrossRef] 11. [CrossRef]
14. Özbek, L. A Study on Adaptive Kalman Filter. ISI’97, Bulletin of the International Statistical Institute, man filter in measuring the output gap. Journal of Proceedings Book 2, 299-300, İstanbul, 1997. Economic Dynamics & Control 2005;29:1611–22.
15. Özbek L, Aliev FA. Comments on “adaptive fad- [CrossRef] ing kalman filter with an application”. Automatica [30] Ozbek L, Efe M. An Adaptive extended kalman 1998;34:1663–4. filter with application to compartment Models.
16. Özbek L. Kalman Filtresi. Ankara: Akademisyen Communications in Statistics-Simulation and Kitabevi; 2016. (Turkish) [CrossRef] Computation 2004;33:145–58. [CrossRef]

Share and Cite