Characteristics of time series

• Expectation, mean & variance
• Covariance & correlation
• Stationarity
• Autocovariance & autocorrelation
• Correlograms
• Cross-correlation

The expectation (\(E\)) of a variable is its mean value in the population

\(\text{E}(x) \equiv\) mean of \(x = \mu\)

We can estimate \(\mu\) from a sample as

\[ m = \frac{\sum_{i=1}^N{x_i}}{N} \]

\(\text{E}([x - \mu]^2) \equiv\) expected deviations of \(x\) about \(\mu\)

\(\text{E}([x - \mu]^2) \equiv\) variance of \(x = \sigma^2\)

We can estimate \(\sigma^2\) from a sample as

\[ s^2 = \frac{1}{N-1}\sum_{i=1}^N{(x_i - m)^2} \]

If we have two variables, \(x\) and \(y\), we can generalize variance

\[ \sigma^2 = \text{E}([x_i - \mu][x_i - \mu]) \]

into covariance

\[ \gamma_{x,y} = \text{E}([x_i - \mu_x][y_i - \mu_y]) \]

If we have two variables, \(x\) and \(y\), we can generalize variance

\[ \sigma^2 = \text{E}([x_i - \mu][x_i - \mu]) \]

into covariance

\[ \gamma_{x,y} = \text{E}([x_i - \mu_x][y_i - \mu_y]) \]

We can estimate \(\gamma_{x,y}\) from a sample as

\[ \text{Cov}(x,y) = \frac{1}{N-1}\sum_{i=1}^N{(x_i - m_x)(y_i - m_y)} \]

Correlation is a dimensionless measure of the linear association between 2 variables, \(x\) & \(y\)

It is simply the covariance standardized by the standard deviations

\[ \rho_{x,y} = \frac{\gamma_{x,y}}{\sigma_x \sigma_y} \]

\[ -1 < \rho_{x,y} < 1 \]

Correlation is a dimensionless measure of the linear association between 2 variables \(x\) & \(y\)

It is simply the covariance standardized by the standard deviations

\[ \rho_{x,y} = \frac{\gamma_{x,y}}{\sigma_x \sigma_y} \]

We can estimate \(\rho_{x,y}\) from a sample as

\[ \text{Cor}(x,y) = \frac{\text{Cov}(x,y)}{s_x s_y} \]

Consider a single value, \(x_t\)

Consider a single value, \(x_t\)

\(\text{E}(x_t)\) is taken across an ensemble of all possible time series

If \(\text{E}(x_t)\) is constant across time, we say the time series is stationary in the mean

Stationarity is a convenient assumption that allows us to describe the statistical properties of a time series.

In general, a time series is said to be stationary if there is

1. no systematic change in the mean or variance
   
2. no systematic trend
   
3. no periodic variations or seasonality

Our eyes are really bad at identifying stationarity, so we will learn some tools to help us

For stationary ts, we define the autocovariance function (\(\gamma_k\)) as

\[ \gamma_k = \text{E}([x_t - \mu][x_{t+k} - \mu]) \]

which means that

\[ \gamma_0 = \text{E}([x_t - \mu][x_{t} - \mu]) = \sigma^2 \]

For stationary ts, we define the autocovariance function (\(\gamma_k\)) as

\[ \gamma_k = \text{E}([x_t - \mu][x_{t+k} - \mu]) \]

"Smooth" series have large ACVF for large \(k\)

"Choppy" series have ACVF near 0 for small \(k\)

For stationary ts, we define the autocovariance function (\(\gamma_k\)) as

\[ \gamma_k = \text{E}([x_t - \mu][x_{t+k} - \mu]) \]

We can estimate \(\gamma_k\) from a sample as

\[ c_k = \frac{1}{N}\sum_{t=1}^{N-k}{(x_t - m)(x_{t+k} - m)} \]

The autocorrelation function (ACF) is simply the ACVF normalized by the variance

\[ \rho_k = \frac{\gamma_k}{\sigma^2} = \frac{\gamma_k}{\gamma_0} \]

The ACF measures the correlation of a time series against a time-shifted version of itself

The autocorrelation function (ACF) is simply the ACVF normalized by the variance

\[ \rho_k = \frac{\gamma_k}{\sigma^2} = \frac{\gamma_k}{\gamma_0} \]

The ACF measures the correlation of a time series against a time-shifted version of itself

We can estimate ACF from a sample as

\[ r_k = \frac{c_k}{c_0} \]

The ACF has several important properties:

• \(-1 \leq r_k \leq 1\)

• \(r_k = r_{-k}\)

• \(r_k\) of periodic function is itself periodic

• \(r_k\) for the sum of 2 independent variables is the sum of \(r_k\) for each of them

Recall the transitive property, whereby

If \(A = B\) and \(B = C\), then \(A = C\)

Recall the transitive property, whereby

If \(A = B\) and \(B = C\), then \(A = C\)

which suggests that

If \(x \propto y\) and \(y \propto z\), then \(x \propto z\)

Recall the transitive property, whereby

If \(A = B\) and \(B = C\), then \(A = C\)

which suggests that

If \(x \propto y\) and \(y \propto z\), then \(x \propto z\)

and thus

If \(x_t \propto x_{t+1}\) and \(x_{t+1} \propto x_{t+2}\), then \(x_t \propto x_{t+2}\)

The partial autocorrelation function (\(\phi_k\)) measures the correlation between a series \(x_t\) and \(x_{t+k}\) with the linear dependence of \(\{x_{t-1},x_{t-2},\dots,x_{t-k-1}\}\) removed

The partial autocorrelation function (\(\phi_k\)) measures the correlation between a series \(x_t\) and \(x_{t+k}\) with the linear dependence of \(\{x_{t-1},x_{t-2},\dots,x_{t-k-1}\}\) removed

We can estimate \(\phi_k\) from a sample as

\[ \phi_k = \begin{cases} \text{Cor}(x_1,x_0) = \rho_1 & \text{if } k = 1 \\ \text{Cor}(x_k-x_k^{k-1}, x_0-x_0^{k-1}) & \text{if } k \geq 2 \end{cases} \]

\[ x_k^{k-1} = \beta_1 x_{k-1} + \beta_2 x_{k-2} + \dots + \beta_{k-1} x_1 \]

\[ x_0^{k-1} = \beta_1 x_1 + \beta_2 x_2 + \dots + \beta_{k-1} x_{k-1} \]

The ACF & PACF will be very useful for identifying the orders of ARMA models

Often we want to look for relationships between 2 different time series

We can extend the notion of covariance to cross-covariance

Often we want to look for relationships between 2 different time series

We can extend the notion of covariance to cross-covariance

We can estimate \(g^{x,y}_k\) from a sample as

\[ g^{x,y}_k = \frac{1}{N}\sum_{t=1}^{N-k}{(x_t - m_x)(y_{t+k} - m_y)} \]

The cross-correlation function is the CCVF normalized by the standard deviations of x & y

\[ r^{x,y}_k = \frac{g^{x,y}_k}{s_x s_y} \]

Just as with other measures of correlation

\[ -1 \leq r^{x,y}_k \leq 1 \]

We often think of correlation in terms of causation

A caution about estimating the ccf in R:

ccf(x, y, ...)

ccf(x, y) estimates the correlation between x[t+k] and y[t]

Thus, I usually

• switch the \(x\) and \(y\) values
  
• consider only negative lags \(k\)