Quantitative Analysis
Parallel Processing
Numerical Analysis
C++ Multithreading
Python for Excel
Python Utilities
Services
Author
Printable PDF file
I. Basic math.
II. Pricing and Hedging.
III. Explicit techniques.
IV. Data Analysis.
1. Time Series.
A. Time series forecasting.
B. Updating a linear forecast.
C. Kalman filter I.
D. Kalman filter II.
E. Simultaneous equations.
2. Classical statistics.
3. Bayesian statistics.
V. Implementation tools.
VI. Basic Math II.
VII. Implementation tools II.
VIII. Bibliography
Notation. Index. Contents.

Time series forecasting.


e are considering a problem of forecasting of a random variable $y$ based on information contained by some vector $x\in R^{M}.$ The $x$ is treated as a sample of some random variable that we also denote as $x$ .

Proposition

We evaluate the quality of the forecast $\hat{y}$ by calculating the quadratic deviation of the forecast: MATH Consequently, the optimal forecast is given by the conditional expectation MATH

Proof

The forecast is some function of $x$ : MATH . Therefore, we require that the function of interest $f_{0}$ would satisfy MATH for any smooth function MATH and a small real number $\varepsilon$ , where the functional $V$ is defined by MATH Thus, we have MATH By properties of conditional expectation, MATH MATH The last equation holds for any smooth $u\left( x\right) $ . Therefore, MATH as claimed.

If the information $x$ is sufficiently large then it makes sense to look for MATH among linear functions: MATH . Repeating the above argument, we have MATH with linear $f_{0}$ and $u$ .

We extend the above requirement to the case of a vector valued variable $y\in R^{N}$ . We require that the function MATH , MATH , for some matrix $A$ , would be such that for any linear MATH , the random quantities MATH and $u\left( x\right) $ are not correlated: MATH We see that the forecasting operation MATH is similar to a linear projection of $y$ on $x$ . This motivates introduction of the operation MATH that takes two vector valued random variables and produces a deterministic matrix. We would like to treat this operation as a scalar product even though it does not have a full set of properties. We will construct a projection operation MATH with the defining properties MATH MATH We proceed to verify that the projection MATH may be defined by the straightforward adaptation of the formula from the elementary geometry MATH on a subclass of random variables with zero mean: $Ex=0,$ $Ey=0.$ Indeed, MATH MATH and MATH MATH MATH MATH

From the two properties, verified here, all the other well known properties of orthogonal projection follow. This allows using of geometric intuition in the computation that follows.

The operation $P(y|x)$ is well defined if the matrix $<x,x>$ is not degenerate. This is certainly so if the random variable $x$ has zero mean because then $<x,x>$ is the covariance matrix of $x$ . However, if $x$ is deterministic then $<x,x>$ is degenerate. Therefore, we represent a random variable $x$ as a sum MATH where $\bar{x}=Ex$ and $\tilde{x}=x-Ex$ . Observe that for two vector valued random variables $x$ and $y$ MATH Therefore, MATH for any $x,y$ . Hence, we extend the definition of MATH to random variables with non-zero mean by orthogonality: MATH MATH Here we used the fact that the MATH must satisfy MATH with $\bar{x}\neq0$ . The only way this can happen is if MATH

Summary

The linear forecast of $y$ based on information $x$ is given by MATH where $\bar{x}=Ex,$ MATH , MATH .

One can verify that this is a maximal likelihood forecast if $x$ and $y$ are jointly normal. To see this it is enough to use a jointly normal distribution function to compute a conditional distribution of $y|x$ . In the section ( Kalman filter II ) we will take such approach.





Notation. Index. Contents.


















Copyright 2007