On the application of the theory of ARMA processes in engineering practice
Below we will say a few words about the discrete-time alternative to mathematical models in the form of linear differential equations, which is generally known but rather unexpected for engineers, namely, autoregressive models - moving average, and very unusual prospects for such modeling, the capabilities of which significantly exceed what you are used to getting from LDU.
The list of potential capabilities of the technology includes analysis of systems with an incoming disturbance inaccessible for observation, determination of the resonance properties of such systems, the spectrum and the process of external excitation, spectral estimation of processes by their short realizations, modeling of the behavior of systems at a low sampling frequency over time, etc.
ARMA processes, well known to economists (more precisely, “econometricians”), are much less known to specialists in automatic regulation, and, in my opinion, are hardly used by mechanical engineers and radioelectronic engineers, especially the “old school” ones. The article attempts to indicate some possible areas of application of the ARMA theory in engineering practice.
In a nutshell, simplified, for those unfamiliar with the subject, about which, in fact, it is. The stochastic continuous-time process x (t) for obvious "digital" reasons "in practice usually matches the discrete-time sequence x [i] with the sampling interval Δt.
In principle, for any process x [i], a representation of the form x [i] - a 1 · x [i-1] - a 2 · x [i-2] - ... - a p is possible
X [ip] = b 0 · f [i] + b 1 · f [i-1] + ... + b q · f [iq] (1),
where a k and b k are constants (for this models) coefficients called the autoregressive-moving average model with the order of autoregression p and the moving average q. or an ARMA (p, q) -model, f [i] is a kind of “incoming” process, about which a little lower. Often (1) is written in a slightly different form (6).
Basically, it's just a digital filter that has both recursive AR and non-recursive MA parts.
There is a correspondence between the ARMA (p, q) models and linear systems (for example, mechanical), for example, described by the well-known linear differential equation of the form
where m, c, k is the mass, stiffness and damping of the mechanical system, f (t) is the external force. The ARMA analogue looks like this:
x [i] - a 1 · x [i-1] - a 2 · x [i-2] = b 1 · f [i-1] (3),
the model coefficients can be quite easy found through the eigenvalues λ 1 and λ 1 * (for brevity, the "oscillating" case is considered) of the linear system and Δt:
a 1 = z + z *, a 2 = - z · z *, b 1 = j (z * -z ) Δt / (2mω 1 ),
where z = exp (λ 1 Δt), λ 1 = -ε 1 + jω 1, j- imaginary unit, * - complex conjugation
for reference:
For the test system, m = 1 kg, c = 100 N / m, k = 0.75 kg / s, Δt = 0.12 s.,
ARMA ( 2.1) -model
x [i] - 0.69433x [i-1] +0.91393 x [i-2] = 0.010696 · f [i-1]
(A very brief explanation of how (2) generally yields (3) The impulse transition function of our linear system, that is, the response of the system to a single impulse:
Record (2) in the “integral” form is called the “convolution” f (t) and h (t), meaning it involves considering the external action as a sequence . elementary pulses recorded in discrete time, well, for example:
C dix x [i], x [i - 1] and x [i-2] in a selected multiplier 1, a 1 and a 2achieve mutual annihilation of infinite “tails” h [i] - on the right side there remain f [i] · h [0] = f [i] · 0 and f [i-1] · h [1] = f [i-1 ] · B 1 . From the point of view of the ARMA theory, the infinite-dimensional model of the moving average MA (∞) is transformed into ARMA (2,1) (although some will say that a purely autoregressive model AR (2) = ARMA (2,0) was obtained by accident.
Remark 1. A reader who is familiar with digital processing of processes will say that just discretizing h (t) is not very correct - it is necessary to limit the function h (t) by frequency to 1 / (2Δt) (filter). Otherwise, there is a frequency masking error. The graphs of the frequency response and phase response of our system, the “analytical” and ARMA models, show why this error in most engineering cases can most likely be neglected (Fig. 1) (here, the frequency response is on a logarithmic scale).
Fig. 1 Frequency response and phase response of the test system.
Remark 2. In practice, the order of the ARMA model can be significantly larger than the example considered above, due to, say, several degrees of freedom of a mechanical system or a complex spectrum of real external influence.
Remark 3. Very important. There are methods (not considered here - many more articles can be written about them) that allow one to estimate the parameters of the ARMA model (namely, the order of the p and q model and the coefficients a k and b k ) only by the resulting process x [i], under the assumption that f [i] is a hypothetical white noise whose variance can also be estimated. In general, such an assessment is the main part of the entire ARMA theory. While these methods do not differ in particular perfection, they are of considerable interest.
Now about why, in fact, all this can (or could) be applied in practice. In addition to the quite obvious - the quick construction of "manually" damped (and undamped) sinusoids from the first two points and two coefficients a 1 and a 2, in my opinion, there are more serious applications of these models in engineering practice.
1. Well, actually, for the simulation of the system’s operation - we give a real external signal f [i] to the input, we get x [i] at the output:
x [i] = a 1 · x [i-1] + a 2 · x [i-2] + ... + a p · x [ip] + b 0 · f [i] + b 1 · f [i-1] + ... + b q · f [iq] (6)
ARMA- the model copes with the task better than the finite-difference model, however, this is noticeable only at large sampling intervals Δt. (in Fig. 2, Δt = 0.12 s (left) and 0.03 s). In what cases it makes sense to mess with ARMA - you decide.
fig. 2. The response of test systems to a single impulse.
2. For spectral estimation, especially when the implementation length of the stationary process is insufficient for observation. Perhaps this is the most famous engineering application of ARMA models. Since a certain digital filter and the variance of the white noise entering it will be obtained for the process under study, the task of constructing an estimate of the PSD is solved in an obvious way. Indeed, it is possible to obtain externally very “smooth” SPM graphs and at the same time create the impression of high resolution. The expected improvement in the assessment is associated with the fact that the researcher brings external information about the nature of the process to the construction of the assessment - usually by setting a known model order.
In short, you need to know how roughly this PSD should look. The “exploratory” studies of this implementation using classical methods can do little, mostly referring to classical (based on FFT) studies of a similar nature, but significantly longer realizations. There is a chance of gross errors.
3. For the analysis of the resonance properties of the system and the spectrum of the external action, in the case when the true external effect is not available for observation. As already noted, it is possible, knowing the process x [i], to determine all the coefficients of the model a k and b k (and the variance of the incoming white noise). Using them, determining the roots of two polynomials with corresponding coefficients, it is easy to find p “poles” and q “zeros” of the model (λ k and μ k) and construct its transfer function - it is possible, probably, not even in the ARMA form (not given here), but in the usual "analytical" form - as we found out above (Fig. 1), the difference is small. For example, for p = 5, q = 3 (so far distracting from the existing, apparently, restrictions on the ratio of p and q), as an option, we have:
Everything is very, very simplified, of course. Based on the well-known nature of the object under study (for example, it is polygon tests of the car’s ride smoothness) and external impact (microprofile of the road), the researcher decided to rewrite the transfer function, for example, like this:
Fig . 3 Analysis of the signal spectrum with highlighting the incoming disturbance
and commented - part of the model associated with λ 1, this is obviously (up to squaring and multiplying by the variance of the incoming hypothetical white noise) the “pink” microprofile of the road (Fig. 3) (that is, we highlighted the unknown spectrum of the real incoming signal - we selected it manually - sort of like “similar "), Λ 2 and λ 3 are the resonant properties of the body on the suspension (possibly longitudinal-angular and vertical modes of vibration). The main problem will, of course, be in determining the parameters of the ARMA model. For the just described, you can sometimes, without any ARMA, "crawl" in the old fashioned way (albeit in electronic form) according to the spectral density graph and measure the peak width at the level of -3 dB, etc., or apply curvitting, sometimes even with great success .
3. For linear prediction x [i]. Apparently, the main application of ARMA is for econometricians. It can be seen from (6) that if the coefficients of the model were estimated using the methods outlined above, the next value x [i] can be estimated accurate to a hypothetical white noise inaccessible to observation b 0 · f [i], the variance of this white noise is estimated together with the coefficients of the model. Usually in this case, dynamic (in real time) adjustment of the model parameters is implied. Apparently, it can be useful in systems of active vibration and noise reduction. TAP specialists know best.
4. To restore an inaccessible process that cannot be observed. When dividing the model into parts, as shown above in Section 3, based on knowledge about the nature of the process under study, it is possible to evaluate separately the spectrum of the incoming perturbation, and separately, the oscillatory properties of the physical system (separate the model into parts). You can go further - create a filter (ARMA-model, the inverse of the original model) that connects the output of the system with the input, and with its help from the resulting process x [i] obtain a temporary implementation of the incoming disturbance. For example, try to restore an undistorted signal recorded with precisely unknown linear distortions by equipment that is not available for a separate study (say, obtained by telemetry).
Based on my humble knowledge, as a result I will express such a subjective judgment. The applicability of ARMA technologies to engineering problems depends heavily on the perfection of methods for estimating the parameters of these models from the resulting signal, or rather, in my opinion, is strongly restrained by the imperfection of these methods. The accumulation of experience in the application of ARMA in engineering seems to make sense, mainly, in anticipation of a very probable “breakthrough” in this area.
The list of potential capabilities of the technology includes analysis of systems with an incoming disturbance inaccessible for observation, determination of the resonance properties of such systems, the spectrum and the process of external excitation, spectral estimation of processes by their short realizations, modeling of the behavior of systems at a low sampling frequency over time, etc.
ARMA processes, well known to economists (more precisely, “econometricians”), are much less known to specialists in automatic regulation, and, in my opinion, are hardly used by mechanical engineers and radioelectronic engineers, especially the “old school” ones. The article attempts to indicate some possible areas of application of the ARMA theory in engineering practice.
In a nutshell, simplified, for those unfamiliar with the subject, about which, in fact, it is. The stochastic continuous-time process x (t) for obvious "digital" reasons "in practice usually matches the discrete-time sequence x [i] with the sampling interval Δt.
In principle, for any process x [i], a representation of the form x [i] - a 1 · x [i-1] - a 2 · x [i-2] - ... - a p is possible
X [ip] = b 0 · f [i] + b 1 · f [i-1] + ... + b q · f [iq] (1),
where a k and b k are constants (for this models) coefficients called the autoregressive-moving average model with the order of autoregression p and the moving average q. or an ARMA (p, q) -model, f [i] is a kind of “incoming” process, about which a little lower. Often (1) is written in a slightly different form (6).
Basically, it's just a digital filter that has both recursive AR and non-recursive MA parts.
There is a correspondence between the ARMA (p, q) models and linear systems (for example, mechanical), for example, described by the well-known linear differential equation of the form
where m, c, k is the mass, stiffness and damping of the mechanical system, f (t) is the external force. The ARMA analogue looks like this:
x [i] - a 1 · x [i-1] - a 2 · x [i-2] = b 1 · f [i-1] (3),
the model coefficients can be quite easy found through the eigenvalues λ 1 and λ 1 * (for brevity, the "oscillating" case is considered) of the linear system and Δt:
a 1 = z + z *, a 2 = - z · z *, b 1 = j (z * -z ) Δt / (2mω 1 ),
where z = exp (λ 1 Δt), λ 1 = -ε 1 + jω 1, j- imaginary unit, * - complex conjugation
for reference:
For the test system, m = 1 kg, c = 100 N / m, k = 0.75 kg / s, Δt = 0.12 s.,
ARMA ( 2.1) -model
x [i] - 0.69433x [i-1] +0.91393 x [i-2] = 0.010696 · f [i-1]
(A very brief explanation of how (2) generally yields (3) The impulse transition function of our linear system, that is, the response of the system to a single impulse:
Record (2) in the “integral” form is called the “convolution” f (t) and h (t), meaning it involves considering the external action as a sequence . elementary pulses recorded in discrete time, well, for example:
C dix x [i], x [i - 1] and x [i-2] in a selected multiplier 1, a 1 and a 2achieve mutual annihilation of infinite “tails” h [i] - on the right side there remain f [i] · h [0] = f [i] · 0 and f [i-1] · h [1] = f [i-1 ] · B 1 . From the point of view of the ARMA theory, the infinite-dimensional model of the moving average MA (∞) is transformed into ARMA (2,1) (although some will say that a purely autoregressive model AR (2) = ARMA (2,0) was obtained by accident.
Remark 1. A reader who is familiar with digital processing of processes will say that just discretizing h (t) is not very correct - it is necessary to limit the function h (t) by frequency to 1 / (2Δt) (filter). Otherwise, there is a frequency masking error. The graphs of the frequency response and phase response of our system, the “analytical” and ARMA models, show why this error in most engineering cases can most likely be neglected (Fig. 1) (here, the frequency response is on a logarithmic scale).
Fig. 1 Frequency response and phase response of the test system.
Remark 2. In practice, the order of the ARMA model can be significantly larger than the example considered above, due to, say, several degrees of freedom of a mechanical system or a complex spectrum of real external influence.
Remark 3. Very important. There are methods (not considered here - many more articles can be written about them) that allow one to estimate the parameters of the ARMA model (namely, the order of the p and q model and the coefficients a k and b k ) only by the resulting process x [i], under the assumption that f [i] is a hypothetical white noise whose variance can also be estimated. In general, such an assessment is the main part of the entire ARMA theory. While these methods do not differ in particular perfection, they are of considerable interest.
Now about why, in fact, all this can (or could) be applied in practice. In addition to the quite obvious - the quick construction of "manually" damped (and undamped) sinusoids from the first two points and two coefficients a 1 and a 2, in my opinion, there are more serious applications of these models in engineering practice.
1. Well, actually, for the simulation of the system’s operation - we give a real external signal f [i] to the input, we get x [i] at the output:
x [i] = a 1 · x [i-1] + a 2 · x [i-2] + ... + a p · x [ip] + b 0 · f [i] + b 1 · f [i-1] + ... + b q · f [iq] (6)
ARMA- the model copes with the task better than the finite-difference model, however, this is noticeable only at large sampling intervals Δt. (in Fig. 2, Δt = 0.12 s (left) and 0.03 s). In what cases it makes sense to mess with ARMA - you decide.
fig. 2. The response of test systems to a single impulse.
2. For spectral estimation, especially when the implementation length of the stationary process is insufficient for observation. Perhaps this is the most famous engineering application of ARMA models. Since a certain digital filter and the variance of the white noise entering it will be obtained for the process under study, the task of constructing an estimate of the PSD is solved in an obvious way. Indeed, it is possible to obtain externally very “smooth” SPM graphs and at the same time create the impression of high resolution. The expected improvement in the assessment is associated with the fact that the researcher brings external information about the nature of the process to the construction of the assessment - usually by setting a known model order.
In short, you need to know how roughly this PSD should look. The “exploratory” studies of this implementation using classical methods can do little, mostly referring to classical (based on FFT) studies of a similar nature, but significantly longer realizations. There is a chance of gross errors.
3. For the analysis of the resonance properties of the system and the spectrum of the external action, in the case when the true external effect is not available for observation. As already noted, it is possible, knowing the process x [i], to determine all the coefficients of the model a k and b k (and the variance of the incoming white noise). Using them, determining the roots of two polynomials with corresponding coefficients, it is easy to find p “poles” and q “zeros” of the model (λ k and μ k) and construct its transfer function - it is possible, probably, not even in the ARMA form (not given here), but in the usual "analytical" form - as we found out above (Fig. 1), the difference is small. For example, for p = 5, q = 3 (so far distracting from the existing, apparently, restrictions on the ratio of p and q), as an option, we have:
Everything is very, very simplified, of course. Based on the well-known nature of the object under study (for example, it is polygon tests of the car’s ride smoothness) and external impact (microprofile of the road), the researcher decided to rewrite the transfer function, for example, like this:
Fig . 3 Analysis of the signal spectrum with highlighting the incoming disturbance
and commented - part of the model associated with λ 1, this is obviously (up to squaring and multiplying by the variance of the incoming hypothetical white noise) the “pink” microprofile of the road (Fig. 3) (that is, we highlighted the unknown spectrum of the real incoming signal - we selected it manually - sort of like “similar "), Λ 2 and λ 3 are the resonant properties of the body on the suspension (possibly longitudinal-angular and vertical modes of vibration). The main problem will, of course, be in determining the parameters of the ARMA model. For the just described, you can sometimes, without any ARMA, "crawl" in the old fashioned way (albeit in electronic form) according to the spectral density graph and measure the peak width at the level of -3 dB, etc., or apply curvitting, sometimes even with great success .
3. For linear prediction x [i]. Apparently, the main application of ARMA is for econometricians. It can be seen from (6) that if the coefficients of the model were estimated using the methods outlined above, the next value x [i] can be estimated accurate to a hypothetical white noise inaccessible to observation b 0 · f [i], the variance of this white noise is estimated together with the coefficients of the model. Usually in this case, dynamic (in real time) adjustment of the model parameters is implied. Apparently, it can be useful in systems of active vibration and noise reduction. TAP specialists know best.
4. To restore an inaccessible process that cannot be observed. When dividing the model into parts, as shown above in Section 3, based on knowledge about the nature of the process under study, it is possible to evaluate separately the spectrum of the incoming perturbation, and separately, the oscillatory properties of the physical system (separate the model into parts). You can go further - create a filter (ARMA-model, the inverse of the original model) that connects the output of the system with the input, and with its help from the resulting process x [i] obtain a temporary implementation of the incoming disturbance. For example, try to restore an undistorted signal recorded with precisely unknown linear distortions by equipment that is not available for a separate study (say, obtained by telemetry).
Based on my humble knowledge, as a result I will express such a subjective judgment. The applicability of ARMA technologies to engineering problems depends heavily on the perfection of methods for estimating the parameters of these models from the resulting signal, or rather, in my opinion, is strongly restrained by the imperfection of these methods. The accumulation of experience in the application of ARMA in engineering seems to make sense, mainly, in anticipation of a very probable “breakthrough” in this area.