The Difference Operator

First differences

\[\begin{eqnarray*} \Delta y_{t} &\equiv& y_{t}-y_{t-1} \end{eqnarray*}\]

Second differences

\[\begin{eqnarray*} \Delta^{2}y_{t}&\equiv&\Delta(\Delta y_{t})\\ &\equiv&\Delta y_{t}-\Delta y_{t-1}\\ &\equiv&(y_{t}-y_{t-1})-(y_{t-1}-y_{t-2})\\ &\equiv&y_{t}-2y_{t-1}+y_{t-2}\\ \end{eqnarray*}\]

Difference Equations and Solutions

  • \(n\)-th order linear difference equation (with constant coefficient) \(\begin{eqnarray*} y_{t}&=&a_{0}+\sum_{i=1}^{n}a_{i}y_{t-i}+X_{t} \end{eqnarray*}\) where \(X_{t}\) is called forcing process. It includes stochastic terms, time trends, and other variables, but not constants and not \(y\), nor lagged values of \(y\).

  • difference form

    \[\begin{eqnarray*} y_{t}&=&a_{0}+\sum_{i=1}^{n}a_{i}y_{t-i}+X_{t}\\ y_{t}&=&a_{0}+a_{1}y_{t-1}+\sum_{i=2}^{n}a_{i}y_{t-i}+X_{t}\\ y_{t}-y_{t-1}&=&a_{0}+a_{1}y_{t-1}-y_{t-1}+\sum_{i=2}^{n}a_{i}y_{t-i}+X_{t}\\ \Delta y_{t}&=&a_{0}+(a_{1}-1)y_{t-1}+\sum_{i=2}^{n}a_{i}y_{t-i}+X_{t}\\ \end{eqnarray*}\]

  • A solution of a D.E shows \(y_{t}\) equal to a function of the x and t, plus perhaps some initial conditions for y, but not lagged values of y. Solutions are not unique.

Iteration

Consider following first-order linear difference equation:

\[\begin{eqnarray*} y_{t}&=&a_{0}+a_{1}y_{t-1}+\varepsilon_{t}, \ \ y_{0}=y_{0} \end{eqnarray*}\]

We know

\[\begin{eqnarray*} y_{1}&=&a_{0}+a_{1}y_{0}+\varepsilon_{1}\\ y_{2}&=&a_{0}+a_{1}y_{1}+\varepsilon_{2}\\ &=&a_{0}+a_{1}(a_{0}+a_{1}y_{0}+\varepsilon_{1})+\varepsilon_{2}\\ &=&a_{0}+a_{1}a_{0}+a_{1}^{2}y_{0}+a_{1}\varepsilon_{1}+\varepsilon_{2}\\ y_{3}&=&a_{0}+a_{1}y_{2}+\varepsilon_{3}\\ &=&a_{0}+a_{1}(a_{0}+a_{1}a_{0}+a_{1}^{2}y_{0}+a_{1}\varepsilon_{1}+\varepsilon_{2})+\varepsilon_{3}\\ &=&a_{0}+a_{1}a_{0}+a_{1}^{2}a_{0}+a_{1}^{3}y_{0}+a_{1}^{2}\varepsilon_{1}+a_{1}\varepsilon_{2}+\varepsilon_{3}\\ &\vdots&\\ y_{t}&=&a_{0}(1+a_{1}+a_{1}^{2}+\cdots+a_{1}^{t-1})+a_{1}^{t}y_{0}+a_{1}^{t-1}\varepsilon_{1}+a_{1}^{t-2}\varepsilon_{2}+\cdots+\varepsilon_{t}\\ &=&a_{0}\sum_{i=0}^{t-1}(a_{1})^{i}+a_{1}^{t}y_{0}+\sum_{i=0}^{t-1}(a_{1})^{i}\varepsilon_{t-i} \end{eqnarray*}\]

Lag Operators

The lag operator \(L\) is defined \(\begin{eqnarray*} L^{i}y_{t}&\equiv& y_{t-i} \end{eqnarray*}\) Lag operator has following properties

  1. \[L(\beta y_{t})=\beta\cdot Ly_{t}\]
  2. \[L(x_{t}+y_{t})=Lx_{t}+Ly_{t}\]
  3. \[(L^{i}+L^{j})y_{t}=L^{i}y_{t}+L^{j}y_{t}\]
  4. \[L^{i}L^{j}y_{t}=L^{i+j}y_{t}=y_{t-i-j}\]
  5. \[L^{-i}y_{t}=y_{t+i}\]
  6. For $$\left a \right <1$$
\[\begin{eqnarray*} \sum_{i=0}^{+\infty}(a^{i}L^{i})y_{t}&=&\frac{y_{t}}{1-aL} \end{eqnarray*}\]
  1. For $$\left a \right >1$$
\[\begin{eqnarray*} \sum_{i=0}^{+\infty}(a^{-i}L^{-i})y_{t}&=&\frac{-aLy_{t}}{1-aL} \end{eqnarray*}\]

First-Order Difference Equations

Consider the first-order difference equation below

\[\begin{eqnarray*} y_{t}&=&\phi y_{t-1}+w_{t} \end{eqnarray*}\]

which can be written using the lag operator as

\[\begin{eqnarray*} y_{t}&=&\phi Ly_{t}+w_{t}\\ y_{t}-\phi Ly_{t}&=&w_{t}\\ (1-\phi L)y_{t}&=&w_{t}\\ (1+\phi L+\phi^{2}L^{2}+\cdots+\phi^{t}L^{t})(1-\phi L)y_{t}&=&(1+\phi L+\phi^{2}L^{2}+\cdots+\phi^{t}L^{t})w_{t}\\ \end{eqnarray*}\]

The compound operator on the left-hand side is

\[\begin{eqnarray*} &&(1+\phi L+\phi^{2}L^{2}+\cdots+\phi^{t}L^{t})(1-\phi L)\\ &=&(1+\phi L+\phi^{2}L^{2}+\cdots+\phi^{t}L^{t})\\ &-&(\phi L+\phi^{2}L^{2}+\cdots+\phi^{t}L^{t}+\phi^{t+1}L^{t+1})\\ &=&1-\phi^{t+1}L^{t+1} \end{eqnarray*}\]

Then we have

\[\begin{eqnarray*} (1-\phi^{t+1}L^{t+1})y_{t}&=&(1+\phi L+\phi^{2}L^{2}+\cdots+\phi^{t}L^{t})w_{t}\\ y_{t}-\phi^{t+1}y_{-1}&=&w_{t}+\phi w_{t-1}+\phi^{2}w_{t-2}+\cdots+\phi^{t}w_{0}\\ y_{t}&=&\phi^{t+1}y_{-1}+w_{t}+\phi w_{t-1}+\phi^{2}w_{t-2}+\cdots+\phi^{t}w_{0} \end{eqnarray*}\]

We have known that

\[\begin{eqnarray*} (1+\phi L+\phi^{2}L^{2}+\cdots+\phi^{t}L^{t})(1-\phi L)y_{t}&=&y_{t}-\phi^{t+1}y_{-1} \end{eqnarray*}\]

if \(|\phi|<1\) and \(y_{-1}\) is finite, \(\phi^{t+1}y_{-1}\) will become negligible as \(t\) becomes large

\[\begin{eqnarray*} (1+\phi L+\phi^{2}L^{2}+\cdots+\phi^{t}L^{t})(1-\phi L)y_{t}&\cong&y_{t} \end{eqnarray*}\]

A sequence \(\{y_{t}\}_{t=-\infty}^{\infty}\) is bounded if there exists a finite number \(\bar{y}\) such that \(|y_{t}|<\bar{y}\) for all \(t\). When \(|\phi|<1\) and when we are considering applying an operator to a bounded sequence, we have

\[\begin{eqnarray*} (1-\phi L)^{-1}&=&\lim_{t\to\infty} (1+\phi L+\phi^{2}L^{2}+\cdots+\phi^{t}L^{t}) \end{eqnarray*}\]

So we have

\[\begin{eqnarray*} (1+\phi L+\phi^{2}L^{2}+\cdots+\phi^{t}L^{t})(1-\phi L)y_{t}&\cong&y_{t}\\ (1-\phi L)^{-1}(1-\phi L)y_{t}&=&y_{t} \end{eqnarray*}\]

By this definition, we have

\[\begin{eqnarray*} (1-\phi L)y_{t}&=&w_{t}\\ y_{t}&=&(1-\phi L)^{-1}w_{t}\\ &=&w_{t}+\phi w_{-1}+\phi^{2}w_{t-2}+\phi^{3}w_{t-3}+\cdots \end{eqnarray*}\]

Second-Order Difference Equations

Consider the second-order difference equation

\[\begin{eqnarray*} y_{t}&=&\phi_{1}y_{t-1}+\phi_{2}y_{t-2}+w_{t}\\ (1-\phi_{1}L-\phi_{2}L^{2})y_{t}&=&w_{t} \end{eqnarray*}\]

The operator in the left-hand side contains a second-order polynomial in the lag operator $L$. Suppose we have

\[\begin{eqnarray*} (1-\phi_{1}L-\phi_{2}L^{2})&=&(1-\lambda_{1}L)(1-\lambda_{2}L)\\ &=&(1-[\lambda_{1}+\lambda_{2}]L+\lambda_{1}\lambda_{2}L^{2}) \end{eqnarray*}\]

Given values for \(\phi_{1}\) and \(\phi_{2}\), we seek numbers \(\lambda_{1}\) and \(\lambda_{2}\) such that

\[\begin{eqnarray*} \begin{cases} \lambda_{1}+\lambda_{2}=\phi_{1}\\ \lambda_{1}\lambda_{2}=-\phi_{2} \end{cases} \end{eqnarray*}\]

Two methods to consider this problem.

  • Method 1

    \[\begin{eqnarray*} (1-\phi_{1}z-\phi_{2}z^{2})&=&(1-\lambda_{1}z)(1-\lambda_{2}z) \end{eqnarray*}\]

    when \(z=\lambda_{1}^{-1}\) or \(z=\lambda_{2}^{-1}\), the right hand side is equal to 0. When

    \[\begin{eqnarray*} \begin{cases} z_{1}=\frac{\phi_{1}-\sqrt{\phi_{1}^{2}+4\phi_{2}}}{-2\phi_{2}}\\ \\ z_{2}=\frac{\phi_{1}+\sqrt{\phi_{1}^{2}+4\phi_{2}}}{-2\phi_{2}} \end{cases} \end{eqnarray*}\]

    the left-hand side is equal to 0 as well. So if we let

    \[\begin{eqnarray*} \begin{cases} \lambda_{1}^{-1}=z_{1}=\frac{\phi_{1}-\sqrt{\phi_{1}^{2}+4\phi_{2}}}{-2\phi_{2}}\\ \\ \lambda_{2}^{-1}=z_{2}=\frac{\phi_{1}+\sqrt{\phi_{1}^{2}+4\phi_{2}}}{-2\phi_{2}} \end{cases} \end{eqnarray*}\]

    both sides are equal to 0.

  • Method 2 It is easy to find that \(\lambda_{1}\) and \(\lambda_{2}\) are roots of equation

    \[\begin{eqnarray*} \lambda^{2}-\phi_{1}\lambda-\phi_{2}&=&0 \end{eqnarray*}\]

    so

    \[\begin{eqnarray*} \begin{cases} \lambda_{1}^{-1}=\frac{\phi_{1}+\sqrt{\phi_{1}^{2}+4\phi_{2}}}{2}\\ \\ \lambda_{2}^{-1}=\frac{\phi_{1}-\sqrt{\phi_{1}^{2}+4\phi_{2}}}{2} \end{cases} \end{eqnarray*}\]

When we have \(\lambda_{1}\) and \(\lambda_{2}\) by the above two methods, we have

\[\begin{eqnarray*} (1-\lambda_{1}L)(1-\lambda_{2}L)y_{t}&=&w_{t}\\ y_{t}&=&(1-\lambda_{1}L)^{-1}(1-\lambda_{2}L)^{-1}w_{t} \end{eqnarray*}\]

If \(\lambda_{1}\neq \lambda_{2}\), we define

\[\begin{eqnarray*} &&(\lambda_{1}-\lambda_{2})^{-1}\left(\frac{\lambda_{1}}{1-\lambda_{1}L}-\frac{\lambda_{2}}{1-\lambda_{2}L}\right)\\ &=&(\lambda_{1}-\lambda_{2})^{-1}\frac{\lambda_{1}(1-\lambda_{2}L)-\lambda_{2}(1-\lambda_{1}L)}{(1-\lambda_{1}L)(1-\lambda_{2}L)}\\ &=&(\lambda_{1}-\lambda_{2})^{-1}\frac{\lambda_{1}-\lambda_{2}}{(1-\lambda_{1}L)(1-\lambda_{2}L)}\\ &=&\frac{1}{(1-\lambda_{1}L)(1-\lambda_{2}L)} \end{eqnarray*}\]

So we have

\[\begin{eqnarray*} y_{t}&=&(1-\lambda_{1}L)^{-1}(1-\lambda_{2}L)^{-1}w_{t}\\ &=&(\lambda_{1}-\lambda_{2})^{-1}\left(\frac{\lambda_{1}}{1-\lambda_{1}L}-\frac{\lambda_{2}}{1-\lambda_{2}L}\right)w_{t}\\ &=&\frac{\lambda_{1}}{\lambda_{1}-\lambda_{2}}\frac{1}{1-\lambda_{1}L}w_{t}-\frac{\lambda_{2}}{\lambda_{1}-\lambda_{2}}\frac{1}{1-\lambda_{2}L}w_{t}\\ &=&\frac{\lambda_{1}}{\lambda_{1}-\lambda_{2}}(1+\lambda_{1}L+\lambda_{1}^{2}L^{2}+\lambda_{1}^{3}L^{3}+\cdots)w_{t}\\ &-&\frac{\lambda_{2}}{\lambda_{1}-\lambda_{2}}(1+\lambda_{2}L+\lambda_{2}^{2}L^{2}+\lambda_{2}^{3}L^{3}+\cdots)w_{t}\\ &=&\frac{\lambda_{1}}{\lambda_{1}-\lambda_{2}}(w_{t}+\lambda_{1}w_{t-1}+\lambda_{1}^{2}w_{t-2}+\lambda_{1}^{3}w_{t-3}+\cdots)\\ &-&\frac{\lambda_{2}}{\lambda_{1}-\lambda_{2}}(w_{t}+\lambda_{2}w_{t-1}+\lambda_{2}^{2}w_{t-2}+\lambda_{2}^{3}w_{t-3}+\cdots)\\ &=&(c_{1}+c_{2})w_{t}+(c_{1}\lambda_{1}+c_{2}\lambda_{2})w_{t-1}+(c_{1}\lambda_{1}^{2}+c_{2}\lambda_{2}^{2})w_{t-2}+\cdots \end{eqnarray*}\]

where \(c_{1}=\lambda_{1}/(\lambda_{1}-\lambda_{2})\) and \(c_{2}=-\lambda_{2}/(\lambda_{1}-\lambda_{2})\).