A Primer on The Unit Circle

A PRIMER ON THE UNIT CIRCLE
with an application to
STATIONARY PROCESSES

A SUPPLEMENT READING FOR
AGECON 993.07: TIME SERIES ANALYSIS WORKSHOP

The Ohio State University, 1998

KRASSIMIR PETROV

This primer is intended to give an overview to the unit circle at an intuitive level without resorting to high-powered mathematics. The concept appears in the context of stochastic processes of particular type – autoregressive processes. When using autoregressive processes, it is of critical importance –  for phisicists, for metereologists, or for economists – to be able to determine solely on the basis of the definition of the process, whether it is stationary or not. It turns out that such a condition indeed exists. Moreover, it is both necessary and sufficient:

Theorem: A random sequence(process)[1] satisfying

(1) xn + b1xn-1 + b2xn-2 + …+ bqxn-q = en,

where en is a white noise process, is stationary if and only if all zeros of

(2) Q(z) = 1 + b1z + b2z2 + … + bqzq

lie outside the unit circle. Moreover, the process is well defined and representable in a moving average form of order infinity.

The theorem needs a detailed explanation. First, a process satisfying (1) is called an autoregressive process of order q, which is denoted by AR(q). If you do not feel comfortable with the notation above, then you may substutute

ak=-bk for all k={1,…,q) to obtain : xn – a1xn-1 – a2xn-2 – … – aqxn-q = en , or equivalently:

(3) xn = a1xn-1 + a2xn-2 + … + aqxn-q + en.

From now on, aj and bj will be consistently used as autoregressive coefficients for (1) and (3) and will be related as above: aj=-bj.

Second, white noise process is understood as conventional. This is a stochastic sequence {et}, tÎZ, whose elements have zero mean, constant variance, and are uncorrelated across time: E(et) = 0,  Var(et) = s2, and E(eiej)=0 “t,i¹j. Above, upper case Z denotes the set of all integers (both positive and negative). One may further impose stronger conditions, e.g., independence instead of uncorrelatedness, or normality of et , but these, of course, are special cases of the above definition, and therefore do not alter its conclusion. For more on white noise, see Hamilton (e.g. p.47-48, and the index).

Three, stationarity is understood in the broad sense: A sequence of random variables y={yt)tÎZ,  with  E(yt)2<µ for all tÎZ, is stationary in the broad sense if  E(yt) = E(y0)  and Cov(yk+t, yk) = Cov(yt, y0) for all kÎZ, i.e., the sequence(process) is stationary if its mean is constant and covariance between elements independent of time.

Four, if and only if indicates that the condition imposed on (2) is both necessary and sufficient for (1) to be stationary.

Five, all zeros of Q(x) means all solutions of the equation Q(x)=0, i.e., all those values of x for which the polynomial has a value of zero. For example, all zeros of the polynomial Q(x)= x2 – 9 are represented by the set {-3, 3}.

Six, lie outside the unit circle is the heart of this primer. The polynomial (2) need not be of degree one. According to the fundamental theorem of algebra, a polynomial of degree q will have exactly q roots (in the language above: exactly q zeroes). However, it is not necessary that these roots be real. Some of them may turn out to be complex. For example, the polynomial Q(x)=x2 + 16 has two roots both of which are complex: {-4i,4i}. The number of complex roots will always be even, and the roots will appear in pairs which are known as complex conjugates. For an easy introduction, or refresfer, on complex numbers see the appendix of the textbook Calculus by  James Stewart.   When the roots are complex, what matters is neither their real, nor their imaginary part, but their absolute value, which is defined as the square root of the sum of the squares of the real and of the imaginary part. Mathematically the absolute value of the number z is denoted by abs(z) and is defined as abs(z)=[Re(z)2 + Im(z)2]1/2. Geometrically, it is represented by the length of the distance from the number to the origin, where the axes of the coordinate system represent the real and the imaginary part of the number. It is namely this geometrical representation of the absolute value of a complex number that gives rise to to the term unit circle: those numbers that have an absolute value less that one, i.e., abs(z)<1, lie inside the unit circle; those numbers that have an absolute value of one, abs(z)=1, lie on the unit cirle; and those numbers that have an absolute value greater than one lie outside the unit circle. Thus, all zeros of Q must lie outside the unit circle should be understood as “all roots of the equation Q=0 must have an absolute value greater than one. At this moment an important note of terminology is in order: there is an intimate relationship between the concepts of absolute value and modulus. Moreover, these have quite the same meaning — both of them represent the Euclidean distance of a number from the origin – but the first is generally used for complex numbers while the second term is used with real numbers. In rare occasions, authors will use only modulus for both real and complex numbers, while others will use absolute value as a universal term; this is usually done in order to emphasize that real numbers are a special case of complex numbers and that the term per se is not important but the concept itself. Thus, one may sometimes see in the econometric literatute authors saying that, for example, all solutions must have a modulus>1. This is to be interpreted that all roots must have an absolute value greater than one, or equivalently, that all roots must lie outside the unit circle. In the usage above, as already mentioned, solutions means roots, and modulus means absolute value – and oftentimes authors use them interchageably in order not to become repetitive

Seven, one should note that whether one uses a’s or b’s as coefficients in the equation is immaterial – the results won’t change and that is natural given the fact that the substitution is equivalent and serves purely notational purposes.

Eight, the process is well defined means that in those cases for which condition (2) is satisfied, equation one defines a unique stationary AR(q) process.

Finally, the theorem says that any stationary autoregressive process can be represented as an infinite moving average process. This is mentioned only as a fact and will not be discussed here.

Almost any serious book on time series analysis will give a detailed rigorous proof of the above theorem. Hamilton’s chapter 3 gives a neat proof based on difference equations. It is hard to give an intuitive insight into why the conclusion of the theorem is so, thus it should suffice to say that (1) as a difference equation has a solution represented in the form of a sum of exponents of the inverses of the roots of the (autoregressive) polynomial (2); in order for the solution of the difference equation to be stable, one must require that the inverses of the roots have an absolute value less than one – this will guarantee that the solution does not “explode”.

As an example, let us consider the simple case of AR(1). Let xn=axn-1 + en.

Then by recursive substitution we obtain that

xn =  a(axn-2 + en-1) + en = a2xn-2 + aen-1 + en =

= a3xn-3 + a2en-2 + aen-1 + en = …

…    = S0µ(ajen-j ).

Thus, it is obvious that the process does not explode if  abs(a)<1, which is the same as the root of  1 – ax=0  Û ax=1 Û x=1/a Þ 1<x=1/a  Þ 1<1/a  Þ    (-1)<a<1.

Now, it is time for you to attempt to solve a problem. Suppose b1=2, b2=1, q=2. This is a second order autoregressive process. If you work the problem out , you will find that the two roots coincide and equal (-1). Therefore this is a process that is called a unit root process. Unit root processes are neither stationary, nor explosive.

If you have understood the theory correctly, you should not have any problems solving within a minute analytically the following two problems:

PROBLEM 1   : b1=    2,   b2=-1.         q=2;

PROBLEM 2   : b1 = -.6,   b2=  -.6;    q=2;

Once you have solved the above two problems, you may open Hamilton’s book on p.17 and look at figure 1.5 . In Hamilton’s notation fj is the same as our aj in our notation. You should be able by mere inspection of the figure to determine whether you have worked correctly.

You assignment is problem 3:

PROBLEM3: b=[-.1, +.3, -.5, +.7, -9], q=5. You are required to rewrite the program given below, which is written in MATLAB, into Gauss and determine whether this process is stationary or not. To doublecheck your work, you are advised to appropriately change the vector q in the Matlab program and to run it.

MATLAB CODE:

% This is a program written by Krassimir Petrov.

% It determines whether an autoregressive stochastic

% sequence is stationary, has a unit root (and possibly

% explosive), or is explosive without any unit roots.

% The process is assumed of the form:

%

% x(n) + b1*x(n-1) + … + bq*x(n-q)=e(n)

%

% and one must solve the polynomial

%

%  1 + b1*z + … + bq*z^q = 0.

%

% The program below solves the problem:

%

% *x(n) – 1.2*x(n-1) + 1.7*x(n-2) – 1.4x(n-3) = e(n)

q=[1 -1.2 1.7 -1.4]

solutions=roots((fliplr(q)))

abs_vals=abs(solutions)

if all(abs_vals>1)

disp(‘the process is stationary’)

elseif any(abs_vals==1)

disp(‘the process has a unit root’)

elseif any(abs_vals<1)

disp(‘the process is explosive’)

end


[1] Here process and sequence are used interchangeably. However, in the literature of stochastic processes it is customary to use sequence only for discrete time processes, and process for continuous time processes. Since here we consider only discrete time, there is no possibility for confusion or misunderstanding.

Leave a Reply