Multivariate state-space models

1. state (process) model

2. observation model

\[ x_{i,t} = x_{i,t-1} + w_{i,t} \\ y_{i,t} = x_{i,t} + a_i + v_{i,t} \]

with

\(w_{i,t} \sim \text{N}(0, q)\)

\(v_{i,t} \sim \text{N}(0, r)\)

\[ x_{1,t} = x_{1,t-1} + w_{1,t} \\ x_{2,t} = x_{2,t-1} + w_{2,t} \\ \vdots \\ x_{n,t} = x_{n,t-1} + w_{n,t} \]

\[ y_{1,t} = x_{1,t} + a_1 + v_{1,t} \\ y_{2,t} = x_{2,t} + a_2 + v_{2,t} \\ \vdots \\ y_{n,t} = x_{n,t} + a_2 + v_{n,t} \\ \]

\[ \begin{bmatrix} x_{1,t} \\ x_{2,t} \\ \vdots \\ x_{n,t} \\ \end{bmatrix} = \begin{bmatrix} x_{1,t-1} \\ x_{2,t-1} \\ \vdots \\ x_{n,t-1} \\ \end{bmatrix} + \begin{bmatrix} w_{1,t} \\ w_{2,t} \\ \vdots \\ w_{n,t} \\ \end{bmatrix} \]

\[ \begin{bmatrix} y_{1,t} \\ y_{2,t} \\ \vdots \\ y_{n,t} \\ \end{bmatrix} = \begin{bmatrix} x_{1,t} \\ x_{2,t} \\ \vdots \\ x_{n,t} \\ \end{bmatrix} + \begin{bmatrix} a_1 \\ a_2 \\ \vdots \\ a_n \\ \end{bmatrix} + \begin{bmatrix} v_{1,t} \\ v_{2,t} \\ \vdots \\ v_{n,t} \\ \end{bmatrix} \]

\[ \mathbf{x}_t = \mathbf{x}_{t-1} + \mathbf{w}_t \\ \mathbf{y}_t = \mathbf{x}_t + \mathbf{a} + \mathbf{v}_t \]

with

\(\mathbf{w}_t \sim \text{MVN}(\mathbf{0}, \mathbf{Q})\)

\(\mathbf{v}_t \sim \text{MVN}(\mathbf{0}, \mathbf{R})\)

No covariance

\[ \mathbf{Q} \stackrel{?}{=} \begin{bmatrix} \sigma & 0 & 0 & 0 \\ 0 & \sigma & 0 & 0 \\ 0 & 0 & \sigma & 0 \\ 0 & 0 & 0 & \sigma \end{bmatrix} ~\text{or}~~ \mathbf{Q} \stackrel{?}{=} \begin{bmatrix} \sigma_1 & 0 & 0 & 0 \\ 0 & \sigma_2 & 0 & 0 \\ 0 & 0 & \sigma_3 & 0 \\ 0 & 0 & 0 & \sigma_4 \end{bmatrix} \]

With covariance

\[ \mathbf{Q} \stackrel{?}{=} \begin{bmatrix} \sigma & \gamma & \gamma & \gamma \\ \gamma & \sigma & \gamma & \gamma \\ \gamma & \gamma & \sigma & \gamma \\ \gamma & \gamma & \gamma & \sigma \end{bmatrix} ~\text{or}~~ \mathbf{Q} \stackrel{?}{=} \begin{bmatrix} \sigma_1 & \gamma_{1,2} & \gamma_{1,3} & \gamma_{1,4} \\ \gamma_{1,2} & \sigma_2 & \gamma_{2,3} & \gamma_{2,4} \\ \gamma_{1,3} & \gamma_{2,3} & \sigma_3 & \gamma_{3,4} \\ \gamma_{1,4} & \gamma_{2,4} & \gamma_{3,4} & \sigma_4 \end{bmatrix} \]

No covariance

\[ \mathbf{R} \stackrel{?}{=} \begin{bmatrix} \sigma & 0 & 0 & 0 \\ 0 & \sigma & 0 & 0 \\ 0 & 0 & \sigma & 0 \\ 0 & 0 & 0 & \sigma \end{bmatrix} ~\text{or}~~ \mathbf{R} \stackrel{?}{=} \begin{bmatrix} \sigma_1 & 0 & 0 & 0 \\ 0 & \sigma_2 & 0 & 0 \\ 0 & 0 & \sigma_3 & 0 \\ 0 & 0 & 0 & \sigma_4 \end{bmatrix} \]

With covariance

\[ \mathbf{R} \stackrel{?}{=} \begin{bmatrix} \sigma & \gamma & \gamma & \gamma \\ \gamma & \sigma & \gamma & \gamma \\ \gamma & \gamma & \sigma & \gamma \\ \gamma & \gamma & \gamma & \sigma \end{bmatrix} \]

Estimating animal locations from tagging is a classic example of applying this type of multivariate state-space model

\[ LAT_t = LAT_{t-1} + w_{LAT,t} \\ LON_t = LON_{t-1} + w_{LON,t} \\ \]

\[ y_t = LAT_t + v_{y,t} \\ x_t = LON_t + v_{x,t} \\ \]

\[ \mathbf{x}_t = \mathbf{x}_{t-1} + \mathbf{w}_t \\ \mathbf{y}_t = \mathbf{x}_t + \mathbf{a} + \mathbf{v}_t \\ \Downarrow \\ \mathbf{x}_t = \mathbf{x}_{t-1} + \mathbf{w}_t \\ \mathbf{y}_t = \mathbf{Z} \mathbf{x}_t + \mathbf{a} + \mathbf{v}_t \]

with \(\mathbf{Z}\) equal to the \(n \times n\) Identity matrix \(\mathbf{I}_n\)

\[ \mathbf{x}_t = \mathbf{x}_{t-1} + \mathbf{w}_t \\ \mathbf{y}_t = \mathbf{Z} \mathbf{x}_t + \mathbf{a} + \mathbf{v}_t \]

Note that both \(\mathbf{x}_t\) and \(\mathbf{y}_t\) are \(n \times 1\) column vectors

What if we had multiple observations of only 1 state?

If so, \(\mathbf{x}_t\) would be a \(1 \times 1\) vector (ie, a scalar) and \(\mathbf{y}_t\) would be an \(n \times 1\) column vector

\[ [x_t] = [x_{t-1}] + [w_t] \]

\[ \begin{bmatrix} y_{1,t} \\ y_{2,t} \\ \vdots \\ y_{n,t} \\ \end{bmatrix} = \begin{bmatrix} x_{1,t} \\ x_{2,t} \\ \vdots \\ x_{n,t} \\ \end{bmatrix} + \begin{bmatrix} a_1 \\ a_2 \\ \vdots \\ a_n \\ \end{bmatrix} + \begin{bmatrix} v_{1,t} \\ v_{2,t} \\ \vdots \\ v_{n,t} \\ \end{bmatrix} \]

\[ [x_t] = [x_{t-1}] + [w_t] \]

\[ \begin{bmatrix} y_{1,t} \\ y_{2,t} \\ \vdots \\ y_{n,t} \\ \end{bmatrix} = \begin{bmatrix} x_{1,t} \\ x_{2,t} \\ \vdots \\ x_{n,t} \\ \end{bmatrix} + \begin{bmatrix} a_1 \\ a_2 \\ \vdots \\ a_n \\ \end{bmatrix} + \begin{bmatrix} v_{1,t} \\ v_{2,t} \\ \vdots \\ v_{n,t} \\ \end{bmatrix} \]

We need a way to map the \(1 \times 1\) state onto the \(n \times 1\) vector of observations

\[ \mathbf{y}_t = ~?~ x_t \]

We'll use the matrix \(\mathbf{Z}\) we saw earlier and treat \(x_t\) like a matrix

\[ \mathbf{y}_t = \mathbf{Z} \mathbf{x}_t \]

Recall that the inner dimensions must match when multiplying matrices, such that

\[ [n \times m] [m \times p] = [n \times p] \]

If \(\mathbf{y}_t\) is \(n \times 1\) and \(\mathbf{x}_t\) is \(1 \times 1\) then \(\mathbf{Z}\) must be \(n \times 1\)

\[ [n \times 1] = [n \times 1] [1 \times 1] \]

For example

\[ \begin{bmatrix} y_{1,t} \\ y_{2,t} \\ y_{3,t} \end{bmatrix} = \begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix} \begin{bmatrix} x_t \end{bmatrix} + \begin{bmatrix} a_1 \\ a_2 \\ a_3 \end{bmatrix} + \begin{bmatrix} v_{1,t} \\ v_{2,t} \\ v_{3,t} \end{bmatrix} \]

What if we had \(n\) observations of \(m\) different states?

We just need to follow the same logic for matrix multiplication

For example

\[ \begin{bmatrix} y_1 \\ y_2 \\ y_3 \\ y_4 \\ y_5 \end{bmatrix}_t = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 1 \\ \end{bmatrix} \times \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}_t + \begin{bmatrix} a_1 \\ a_2 \\ a_3 \\ a_4 \\ a_5 \end{bmatrix} + \begin{bmatrix} v_1 \\ v_2 \\ v_3 \\ v_4 \\ v_5 \end{bmatrix}_t \]

\(y_1\) observes \(x_1\)
\(y_2\) & \(y_3\) observe \(x_2\)
\(y_4\) & \(y_5\) observe \(x_3\)

We can use the same approach to systematically evaluate the data support for different hypotheses about the underlying population structure

\[ \mathbf{Z} = \begin{bmatrix} 1 \\ 1 \\ \vdots \\ 1 \end{bmatrix} ~~ \text{vs} ~~ \mathbf{Z} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ \vdots & \vdots & \vdots \\ 0 & 0 & 1 \end{bmatrix} ~~ \text{vs} ~~ \mathbf{Z} = \begin{bmatrix} 1 & 0 & \dots & 0 \\ 1 & 0 & \dots & 0 \\ \vdots & \vdots & \ddots & 0 \\ 0 & 0 & \dots & 1 \end{bmatrix} \]

Here is an example of using different forms of \(\mathbf{Z}\) to evaluate the possible metapopulation structure of Harbor Seals

Harbor seal example

Number today is a function of the number yesterday

and the number of predators, prey & competitors

and external forces at various times

\[ x_{i,t} = \sum^{m}_{j = 1}{b_{i,j} x_{j,t}} + w_t \]

where \(b_{i,j}\) is the effect of the \(j^{th}\) species on species \(i\)

\(b_{i,j}\) with \(i = j\) is the density-dependent effect

We can write this model in matrix notation as

\[ \mathbf{x}_t = \mathbf{B} \mathbf{x}_{t-1} + \mathbf{w}_t \]

with

\[ \mathbf{B} = \begin{bmatrix} b_{1,1} & b_{1,2} & \dots & b_{1,m} \\ b_{2,1} & b_{1,2} & \dots & b_{2,m} \\ \vdots & \vdots & \ddots & \vdots \\ b_{m,1} & b_{m,2} & \dots & b_{m,m} \end{bmatrix} \]

Including the effects of exogenous drivers yields

\[ \mathbf{x}_t = \mathbf{B} \mathbf{x}_{t-1} + \mathbf{C} \mathbf{c}_{t-k} + \mathbf{w}_t \]

Note that the lag \(k \geq 0\)

Including the effects of exogenous drivers

\[ \mathbf{x}_t = \mathbf{B} \mathbf{x}_{t-1} + \mathbf{C} \mathbf{c}_{t-k} + \mathbf{w}_t \]

The \(m \times p\) matrix \(\mathbf{C}\) contains the effect(s) of each covariate (cols) on each state (rows)

The \(p \times 1\) column vector \(\mathbf{c}_{t-k}\) contains each of the \(p\) covariates at time \(t - k\)

The effect(s) of covariates can vary by state/species/etc

\[ \mathbf{C} = \begin{bmatrix} C_{1, Temp} & C_{1, DO} \\ C_{2, Temp} & C_{2, DO} \\ \vdots & \vdots \\ C_{m, Temp} & C_{m, DO} \end{bmatrix} ~~ \text{or} ~~ \mathbf{C} = \begin{bmatrix} C_{Temp} & C_{DO} \\ C_{Temp} & C_{DO} \\ \vdots & \vdots \\ C_{Temp} & C_{DO} \end{bmatrix} \]

with

\[ \mathbf{c}_{t-k} = \begin{bmatrix} Temp_{t-k} \\ DO_{t-k} \end{bmatrix} \]

Here is an example of estimating species interactions from time series data

Holmes, Ward & Scheuerell (2018) Chap 13