Backshift notation
When we learned linear regression techniques, it was convenient to move away from an arithmetic expansion such as:
\[y_i = \beta_0 + \beta_1 x_{1,i} + \beta_2 x_{2,i} + \beta_k x_{k,i} + \varepsilon_i\]
And instead move toward a matrix representation, such as:
\[\boldsymbol{y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\varepsilon}\]
We will now learn a similar technique for time series analysis, which collapses all lags and their coefficients using the backshift operator.
The backshift operator
Let \(\boldsymbol{Y}\) be a time series random variable observed at regular time periods \(T = \{1, 2, \ldots, n\}\). Then the unary operator \(B\) (backshift) is defined as follows:
\[B \, Y_t = Y_{t-1}\]
\[B^k \, Y_t = Y_{t-k}\]
Written thus, we can treat \(B\) almost as a separate variable,1 and use it to remove lagged terms from our equations:
\[\begin{aligned} & \, Y_t = 1.3\,Y_{t-1} - 0.36\,Y_{t-2} + \omega_t \\ \\ \Longrightarrow & \,Y_t = 1.3\,B\,Y_t - 0.36\,B^2\,Y_t + \omega_t \\ \\ \Longrightarrow & \, Y_t = Y_t(0.5B - 0.36B^2) + \omega_t \\ \\ \Longrightarrow & \,\omega_t = Y_t(1 - 1.3B + 0.36B^2) \end{aligned}\]
The model above was an AR(2) process, but we can equally use this notation with MA models:
\[\begin{aligned} & \, Y_t = \omega_t + 0.8\,\omega_{t-1} + 0.16\,\omega_{t-2} \\ \\ \Longrightarrow & \,Y_t = \omega_t + 0.8\,B\,\omega_t + 0.16\,B^2\,\omega_t \\ \\ \Longrightarrow & \, Y_t = \omega_t(1 + 0.8B + 0.16B^2) \end{aligned}\]
The AR and MA characteristic polynomials
So far, this hasn’t seemed to save much space or offer us any new possibilities. We can bring it altogether by introducing two characteristic polynomials. These equations aren’t any sort of theorem or result, just a definition:
The characteristic polynomial of an AR process of order p is defined as follows:
\[\Phi(B) = 1 - \phi_1 B - \phi_2 B^2 - ... - \phi_p B^p\]
The characteristic polynomial of an MA process of order q is defined as follows:
\[\Theta(B) = 1 + \theta_1 B + \theta_2 B^2 + ... + \theta_q B^q\]
With these two “shortcuts” we can represent every AR process as follows:
\[\omega_t = \Phi(B) \cdot Y_t\]
And we can represent every MA process as follows:
\[Y_t = \Theta(B) \cdot \omega_t\]
Roots of the characteristic polynomials
In the examples I gave above, we could say that for the AR(2) process,
\[\Phi(B) = 1 - 1.3B + 0.36B^2\]
Keen-eyed mathemagicians might have already noticed that we can factor and solve for the roots of this equation:
\[\Phi(B) = (1 - 0.4B)(1 - 0.9B)\]
\[\Phi(B) = 0 \iff B \in \{2.5, 1.111\ldots\}\]
Likewise, we could solve for the roots of the MA(2) characteristic polynomial
\[\Theta(B) = 1 + 0.8B + 0.16B^2\]
\[\Theta(B) = (1 + 0.4B)^2\]
\[\Theta(B) = 0 \iff B = -2.5\]
The dramatic reveal of why we would want to solve the roots of these polynomials must wait for the next page.
Not a random variable, but a variable we could solve for, such as \(x\) in the equation \(x^2 - 4x = -3\)↩︎