Stationary processes

Let $X$, $Y$, and $Z$ be independent random variables with expectation 1 and variance 1. Compute

What is the correlation coefficient, $\operatorname {Corr}\left[ X_1, X_2 \right]$, between the random variables $X_1$ and $X_2$ if $\operatorname {Var}\left[X_1\right]=5$, $\operatorname {Var}\left[X_2\right]=10$, and $\operatorname {Cov}\left[ X_1, X_2 \right]=-4.5$?

Let $e_0,e_1,\ldots$ be a sequence of independent, identically distributed random variables with mean $m$ and variance $\sigma^2$. Let $\left\{ X_t, t=1,\ldots \right\}$ be the stochastic process defined by $X_t=1.2e_t+0.9e_{t-1}$. Compute the mean $m_X(t)=\mathbb {E}\left[X(t)\right]$, and the covariance function $r_X(s,t)=\operatorname {Cov}\left[ X_s, X_t \right]$. Show that $\left\{ X_t\right\}$ is weakly stationary.

Let $X_1,X_2,\ldots$ be a sequence of zero-mean, independent, identically distributed random variables with variance $\sigma^2$. Define

Prove that $\left\{ Y_k\right\}$ is non-stationary by computing $r_Y(1,2)$ and $r_Y(2,3)$. The non-stationarity can be corrected by letting $Y_1=cX_1$ for an appropriate choice of the constant $c$. Find such a $c$.

Find a physically realistic explanation to the dependence structure at lag $\tau =9$ in the figure below, which shows the correlation function of monthly mean temperature in Stockholm (left) and a scatter plot of mean temperatures taken nine months apart (right). The correlation is almost 0 for $\tau =9$ despite a strong non-linear dependence.

#| echo: false
#| fig-cap: "Left: Correlation function of monthly mean temperature in Stockholm. Right: Scatter plot of mean temperatures taken nine months apart."
exec(open("figs/figs_recreated/2/004.tex.py").read())

Let $Y$ and $Z$ be two independent random variables with expected values $0$ for both and variances $\sigma_y^2$ and $\sigma_z^2$. A stochastic process is created from

Is the process weakly stationary?

Let $f_1,\ldots ,f_n$ be different (non-random) frequencies, and let $A_1,\ldots ,A_n$ and $\phi_1,\ldots ,\phi_n$ be mutually independent random variables, with all $\phi_k$ uniform on $[0,2\pi ]$. Let $A_0$ be a random constant, independent of all other $A_k$ and $\phi_k$. Consider the process

and the time-averaged power $E_T=\frac{1}{T}\int_0^T X(t)^2\, \mathrm{d}t$ over the interval $[0,T]$. Expanding $X(t)^2$ by trigonometric identities gives

It is stated to be "easy to see" that $T^{-1}\int_0^T\cos (2\pi f_k t+\phi_k)\, \mathrm{d}t \to 0$ as $T\to \infty$, and also, for $f_k \neq f_l$,

so that, as $T\to \infty$,

Complete the "easy to see" part of this convergence, i.e., prove the three limit statements above.

A physicist studies the amount of carbon dioxide, $W(t)$, at time $t$, near a highly trafficked road and models the amount by a stationary process. During one day she gets the measurements $w(1),w(2),\ldots ,w(50)$, which she regards as a realization of the stationary stochastic process $W(t)$. In order to study the dependence between $W(t)$, $W(t+1)$, and $W(t+2)$ she makes two scatter plots at time lags 1 and 2 of the observed data, that is, she plots the points $(w(t),w(t+1))$ and $(w(t),w(t+2))$; see the figure below.

#| echo: false
#| fig-cap: "Scatter plots of pairs of measurements in Exercise 2:8."
exec(open("figs/figs_recreated/2/001.tex.py").read())

What can you tell from them? She has to choose amongst the following two different stochastic processes:

where $e(t)$ are zero-mean, independent, identically distributed random variables with variance 1. Which one should she choose?

Which of the following functions can be the covariance functions to a stationary process and which can not?

A digital image consists of pixels, $M(x,y)$, arranged in a large matrix. Each pixel takes a real value, which determines its color. The image captured by a digital camera is often affected by noise if the light is dim. In order to reduce the effect of the noise we replace each pixel value of the image with the average of the original pixel value and the four closest pixel values:

Using statistical terminology, we might say that we estimate the ``true'' pixel value with the average of the outcomes of five (possibly correlated) random variables. Your task is to compute the variance of this estimate. We consider the image as a stochastic process:

where $S(x,y)$ is a constant (the ``true'' image) and $N(x,y)$ is correlated noise with covariance $\operatorname {Cov}\left[ N(x_1,y_1), N(x_2,y_2) \right] = \rho^{\delta }\sigma^2$, where $\rho =0.5$, $\sigma =1$, and $\delta$ is the ordinary euclidian distance between the pixel $(x_1,y_1)$ and $(x_2,y_2)$. Using this model, compute the variance $\operatorname {Var}\left[\widehat{M}(x,y)\right]$.

Let $Y$ be a random variable such that $\mathbb {P}\left(Y=-1\right)=\mathbb {P}\left(Y=1\right)=0.5$. Let $Z$ be a random variable uniformly distributed in the interval $[0,2\pi )$.

A weakly stationary process has expected value $m$ and covariance function $r(0)=3$, $r(\pm 1)=-1$ and zero for all other values.

We are interested in a low frequency signal, but unfortunately our bias-free measurements of the signal $\left\{ X(n)\right\}$ are disturbed by noise. We model our recorded signal as a deterministic low frequency component (the one we are interested in) plus a stationary stochastic process (the noise). We have reasons to believe that the covariance function, $r(\tau )$, of the noise can be approximated with

In order to suppress the noise, we consider the following two options:

where $\left\{ Y_N(n)\right\}$ and $\left\{ Z_M(n)\right\}$ are two different approximations of the low frequency signal of our interest.

A division of the national Swedish Institute for Building Research wants to estimate the average noise level in a specific area. The dependence between measurements, $\left\{ Y_j\right\}$, at different locations is described by $\operatorname {Cov}\left[ Y_i, Y_j \right] = r_Y(d) = e^{-d}$, where $d$ is the distance between the $i$:th and the $j$:th measurement point. They are restricted to estimate the noise level with the average of the sound level recorded by three microphones. They discuss two different placements, shown in the figure below. Do some computations, and argue in favor of one of the alternatives.

#| echo: false
#| fig-cap: "Two different microphone placements."
exec(open("figs/figs_recreated/2/002.tex.py").read())

A zero-mean sequence, $\left\{ X_k\right\}$, has covariance function

We would like to estimate $m$ by observing $T$ consecutive samples of the process. How large must $T$ be, in order for the standard deviation of the estimate to be less than $0.1$? You can make appropriate approximations.

Suppose that $\left\{ X_k\right\}$ is a stationary process with unknown mean $m$, known variance $\sigma^2$, and correlation function $r(\tau ) = 0.5^{\left|\tau \right|}$. The mean $m$ is estimated with $\frac{1}{N}\sum_1^N X_k$. Suppose that $N$ is large and approximate the variance of the estimator. Also, find the number of elements that would have been necessary in order to achieve the same variance, if the elements of $\left\{ X_k\right\}$ had been uncorrelated.

In Example 2.12 of the source text, the mean $m$ of a stationary sequence $\left\{ X_t\right\}$ is estimated by the average $\widehat{m}_n = \overline{x} = \frac{1}{n}\sum_1^n x_k$ of observations of $X_1,\ldots ,X_n$, where the covariance function is $r(\tau ) = \sigma^2\theta^{\left|\tau \right|}$ for some $0 < \theta < 1$ (writing $\theta =e^{-\alpha }$). For large $n$, the asymptotic standard deviation of $\widehat{m}_n$ is found to be

The example states the following general rule of thumb: if one wants to get the same variance of the mean in a sample from a stochastic process with this covariance function as in a sample of independent random variables with the same variance, then one has to increase the number of observations by a factor $(K^{1/\tau_K}+1)/(K^{1/\tau_K}-1)$, where $\tau_K$ is the time lag where the correlation has decreased from one to $1/K$. Motivate the rule and show that it is exact if the correlation function is of the form $\rho (\tau ) = \theta^{\left|\tau \right|}$.

The Swedish Meteorological and Hydrological Institute (SMHI) reports daily ozone levels for a number of Swedish cities. We are particularly interested in the measurements during July, which seems like an ``ordinary'' month, with mean 335 and standard deviation 16. We assume that the observations, $x_1,\ldots ,x_{31}$, come from a stationary Gaussian process, $\left\{ X_t\right\}$. The covariance function is stated below for a few time lags,

Suppose that we want to estimate the mean value, using only three measurements during July. Which measurements should we choose? Answer the question by computing the variance of

[Figure omitted -- see figs/figs_original/2/003.png]

Example 2.11 of the source text considers 40 samples of 25 dependent AR(1)-observations, $X_{t+1} = 0.9X_t+e_t$, where the $e_t$ are independent normal variables, compared against 40 samples of 25 independent normal observations $Y_t$ with $\operatorname {Var}\left[Y_t\right]=\operatorname {Var}\left[X_t\right]=1$ and $\mathbb {E}\left[Y_t\right]=\mathbb {E}\left[X_t\right]=0$. Although the observations within each dependent AR(1) sample are less spread out than those in the independent samples, the observed sample averages are more variable for the AR(1)-process: the calculated standard deviation of the sample average is about $0.68$ for the dependent samples, versus about $0.99$ for the independent samples -- i.e., the more spread-out sample actually gives better precision for estimating the mean.

Repeat the experiment with the model $X_{t+1} = -0.9X_t+e_t$ to see how negative correlation affects the estimate.

Let $Y$ and $\left\{ X_n, n=1,2,\ldots \right\}$ be independent normal variables and define $Z_n = X_n+Y$. Is $\left\{ Z_n, n=1,2,\ldots \right\}$ a linearly ergodic sequence? Compare with the covariance criterion for ergodicity.