Stationary processes

Let $\ldots , Z_{-1}, Z_{0}, Z_{1}, Z_{2}, \ldots$ be independent real random variables with means 0 and variances $1$, and let $\alpha , \beta \in \mathbb {R}$. Show that there exists a (weakly) stationary sequence $\left\{ W_{n}\right\}$ satisfying $W_{n} = \alpha W_{n-1}+\beta W_{n-2}+Z_{n}, n = \ldots ,-1,0,1, \ldots$, if the (possibly complex) zeros of the quadratic equation $z^{2}-\alpha z-\beta = 0$ are smaller than 1 in absolute value.

Let $U$ be uniformly distributed on $[0,1]$ with binary expansion $U = \sum_{i = 1}^{\infty } X_{i} 2^{-i}$. Show that the sequence

is strongly stationary, and calculate its autocovariance function.

Let $\left\{ X_{n}: n = \ldots ,-1,0,1, \ldots \right\}$ be a stationary real sequence with mean 0 and autocovariance function $c(m)$.

(i) Show that the infinite series $\sum_{n = 0}^{\infty } a_{n} X_{n}$ converges almost surely, and in mean square, whenever $\sum_{n = 0}^{\infty }\left|a_{n}\right| < \infty$.

(ii) Let

where $\sum_{k = 0}^{\infty }\left|a_{k}\right| < \infty$. Find an expression for the autocovariance function $c_{Y}$ of $Y$, and show that

Let $X = \left\{ X_{n}: n \geq 0\right\}$ be a discrete-time Markov chain with countable state space $S$ and stationary distribution $\mathbf{\pi }$, and suppose that $X_{0}$ has distribution $\mathbf{\pi }$. Show that the sequence $\left\{ f\left(X_{n}\right): n \geq 0\right\}$ is strongly stationary for any function $f: S \rightarrow \mathbb {R}$.

Let $W, X, Y, Z$ have a multivariate normal distribution with zero means and covariance matrix

Let $U = W+i X$ and $V = Y+i Z$. Show that $U$ and $V$ are uncorrelated but not independent. Why does this not violate the conclusion of Example (4.5.9) that multivariate normal random variables are independent if and only if they are uncorrelated?

Let $X_{n} = \cos (n S+U)$ for $n \in \mathbb {Z}$, where $U$ is uniformly distributed on $(-\pi , \pi )$, and $S$ is independent of $U$ with a density function $g$ that is symmetric on its support $(-\pi , \pi )$. Show that $X = \left\{ X_{n}: n \in \mathbb {Z}\right\}$ is weakly stationary, and find its autocorrelation function $\rho_{X}$.

Let $Y_{n} = Z_{n}+a Z_{n-1}$ for $n \in \mathbb {Z}$, where $\left|a\right| < 1$ and the $Z_{n}$ are independent, identically distributed random variables with means 0 and variances 1. Is it possible to choose $g$ in such a way that $Y = \left\{ Y_{n}: n \in \mathbb {Z}\right\}$ has the same autocorrelation function as $X$? If so, how?

Let $X$ be a (weakly) stationary sequence with zero mean and autocovariance function $c(m)$.

(i) Find the best linear predictor $\widehat{X}_{n+1}$ of $X_{n+1}$ given $X_{n}$.

(ii) Find the best linear predictor $\widetilde{X}_{n+1}$ of $X_{n+1}$ given $X_{n}$ and $X_{n-1}$.

(iii) Find an expression for $D = \mathbb {E}\left[\left(X_{n+1}-\widehat{X}_{n+1}\right)^{2}\right]-\mathbb {E}\left[\left(X_{n+1}-\widetilde{X}_{n+1}\right)^{2}\right]$, and evaluate this expression when:

Does there exist a (weakly) stationary sequence $\left\{ X_{n}:-\infty < n < \infty \right\}$ with zero means and autocovariance function

Assuming that such a sequence exists, find the best linear predictor $\widehat{X}_{n}$ of $X_{n}$ given $X_{n-1}$, $X_{n-2}, \ldots$, and show that the mean squared error of prediction is $\left(1+a^{2}\right)^{-1}$. Verify that $\left\{ \hat{X}_{n}\right\}$ is (weakly) stationary.

Let $X_{n} = A \cos (n \lambda )+B \sin (n \lambda )$ where $A$ and $B$ are uncorrelated random variables with zero means and unit variances. Show that $X$ is stationary with a spectrum containing exactly one point.

Let $U$ be uniformly distributed on $(-\pi , \pi )$, and let $V$ be independent of $U$ with distribution function $F$. Show that $X_{n} = e^{i(U-V n)}$ defines a stationary (complex) sequence with spectral distribution function $F$.

Find the autocorrelation function of the stationary process $\left\{ X(t):-\infty < t < \infty \right\}$ whose spectral density function is:

(i) $N(0,1)$,

(ii) $f(x) = \frac{1}{2} e^{-\left|x\right|},-\infty < x < \infty$.

Let $X_{1}, X_{2}, \ldots$ be a real-valued stationary sequence with zero means and autocovariance function $c(m)$. Show that

where $F$ is the spectral distribution function. Deduce that $n^{-1} \sum_{j = 1}^{n} X_{j} \xrightarrow {\text{ m.s. }} 0$ if and only if $F(0)-F(0-) = 0$, and show that

Let $Z = \left\{ Z_{n}: n \in \mathbb {Z}\right\}$ be a sequence of uncorrelated random variables with means 0 and variances 1, and suppose the sequence $X = \left\{ X_{n}: n \in \mathbb {Z}\right\}$ satisfies $X_{n} = \phi X_{n-1}+Z_{n}+\theta Z_{n-1}$.

Let $Z = \left\{ Z_{n}: n \in \mathbb {Z}\right\}$ be a sequence of uncorrelated random variables with means 0 and variances $1$, and suppose $X_{n} = \phi X_{n-1}+\theta X_{n-2}+Z_{n}$ for $n \in \mathbb {Z}$, where $\phi^{2}+4 \theta > 0$. Suppose $d_{k}$ is a real sequence such that

Show that $d_{k} = a_{1} r_{1}^{k}+a_{2} r_{2}^{k}$, where $a_{1}+a_{2} = 1$ and $a_{1} r_{1}+a_{2} r_{2} = \phi$. Show further that

Hence find $\mathbb {E}\left[X_{n}^{2}\right]$ in terms of $\phi$ and $\theta$ when $\left|r_{1}\right|,\left|r_{2}\right| < 1$.

Let $X = \left\{ X_{n}: n \in \mathbb {Z}\right\}$ be a stationary process with zero means and variances $\sigma^{2}$, and let $\widehat{X}_{n+1}$ be the best linear predictor of $X_{n+1}$ given $\left\{ X_{r}: r \leq n\right\}$. (See Section 9.2.) The Kolmogorov-Szegő formula states that

where $f$ is the spectral density of $X$. Verify this for the autoregressive scheme of Example (9.3.23).

Let $S$ be the spectral process of a stationary process $X$ with zero mean and unit variance. Show that the increments of $S$ have zero means.

Let $X$ be a discrete-time stationary process having zero means, continuous strictly positive spectral density function $f$, and with spectral process $S$. Let

Show that $\ldots , Y_{-1}, Y_{0}, Y_{1}, \ldots$ is a sequence of uncorrelated random variables with zero means and unit variances.

Show that $X_{n}$ may be represented as a moving average $X_{n} = \sum_{j = -\infty }^{\infty } a_{j} Y_{n-j}$ where the $a_{j}$ are constants satisfying

Let $X$ be a discrete-time stationary sequence with zero mean and unit variance, and whose fdds are of the multivariate-normal type. Show that the spectral process of $X$ has independent increments having normal distributions.

Let $T = \left\{ 1,2, \ldots \right\}$ and let $\ell$ be the set of invariant events of $\left(\mathbb {R}^{T}, \mathscr {B}^{T}\right)$. Show that $\ell$ is a $\sigma$-field.

Assume that $X_{1}, X_{2}, \ldots$ is a stationary sequence with autocovariance function $c(m)$. Show that

Assuming that $j^{-1} \sum_{i = 0}^{j-1} c(i) \rightarrow \sigma^{2}$ as $j \rightarrow \infty$, show that

Let $X_{1}, X_{2}, \ldots$ be independent identically distributed random variables with zero mean and unit variance. Let

where the $\alpha_{i}$ are constants satisfying $\sum_{i} \alpha_{i}^{2} < \infty$. Use the martingale convergence theorem to show that the above summation converges almost surely and in mean square. Prove that $n^{-1} \sum_{i = 1}^{n} Y_{i} \rightarrow 0$ a.s. and in mean, as $n \rightarrow \infty$.

Show that the function $c(s, t) = \min \left\{ s, t\right\}$ is positive definite. That is, show that

for all $0 \leq t_{1} < t_{2} < \cdots < t_{n}$ and all complex numbers $z_{1}, z_{2}, \ldots , z_{n}$ at least one of which is non-zero.

Let $X_{1}, X_{2}, \ldots$ be a stationary Gaussian sequence with zero means and unit variances which satisfies the Markov property. Find the spectral density function of the sequence in terms of the constant $\rho = \operatorname {Cov}\left[X_{1}, X_{2}\right]$.

Show that a Gaussian process is strongly stationary if and only if it is weakly stationary.

Let $X$ be a stationary Gaussian process with zero mean, unit variance, and autocovariance function $c(t)$. Find the autocovariance functions of the processes

Let $X(s, t)$ be a two-parameter, zero-mean, Gaussian process on the positive quadrant $[0, \infty )^{2}$ of $\mathbb {R}^{2}$, with covariance function $\operatorname {Cov}\left[X(s, t), X(u, v)\right] = (s \wedge u)(t \wedge v)$, where $x \wedge y = \min \left\{ x, y\right\}$. For any rectangle $R$ with corners $(s, t),(u, t),(s, v),(u, v)$ where $0 \leq s < u < \infty$ and $0 \leq t < v < \infty$, define

Find $\operatorname {Var}\left[Y(R)\right]$, and show that $Y$ has independent increments.

Let $\ldots , X_{-1}, X_{0}, X_{1}, \ldots$ be uncorrelated random variables with zero means and unit variances, and define

where $\alpha$ and $\beta$ are constants satisfying $\left|\beta \right| < 1,\left|\beta -\alpha \right| < 1$. Find the best linear predictor of $Y_{n+1}$ given the entire past $Y_{n}, Y_{n-1}, \ldots$

Let $\left\{ Y_{k}:-\infty < k < \infty \right\}$ be a stationary sequence with variance $\sigma_{Y}^{2}$, and let

where $a_{0}, a_{1}, \ldots , a_{r}$ are constants. Show that $X$ has spectral density function

where $f_{Y}$ is the spectral density function of $Y, \sigma_{X}^{2} = \operatorname {Var}\left[X_{1}\right]$, and $G_{a}(z) = \sum_{k = 0}^{r} a_{k} z^{k}$. Calculate this spectral density explicitly in the case of 'exponential smoothing', when $r = \infty$, $a_{k} = \mu^{k}(1-\mu )$, and $0 < \mu < 1$.

Suppose that $\widehat{Y}_{n+1} = \alpha Y_{n}+\beta Y_{n-1}$ is the best linear predictor of $Y_{n+1}$ given the entire past $Y_{n}, Y_{n-1}, \ldots$ of the stationary sequence $\left\{ Y_{k}:-\infty < k < \infty \right\}$. Find the spectral density function of the sequence.

Meteorites fall at integer times $T_{1}, T_{2}, \ldots$ where $T_{n} = X_{1}+X_{2}+\cdots +X_{n}$. We assume that the $X_{i}$ are independent, $X_{2}, X_{3}, \ldots$ are identically distributed, and the distribution of $X_{1}$ is such that the probability that a meteorite falls at time $n$ is constant for all $n$. Let $Y_{n}$ be the indicator function of the event that a meteorite falls at time $n$. Show that $\left\{ Y_{n}\right\}$ is stationary and find its spectral density function in terms of the characteristic function of $X_{2}$.

Let $X = \left\{ X_{n}: n \geq 1\right\}$ be given by $X_{n} = \cos (n U)$ where $U$ is uniformly distributed on $[-\pi , \pi ]$. Show that $X$ is stationary but not strongly stationary. Find the autocorrelation function of $X$ and its spectral density function.

Let $Z_{1}, Z_{2}, \ldots$ be uncorrelated variables, each with zero mean and unit variance.

Show that the complex-valued stationary process $X = \left\{ X(t):-\infty < t < \infty \right\}$ has a spectral density function which is bounded and uniformly continuous whenever its autocorrelation function $\rho$ is continuous and satisfies $\int_{0}^{\infty }\left|\rho (t)\right| d t < \infty$.

Let $X = \left\{ X_{n}: n \geq 1\right\}$ be stationary with constant mean $\mu = \mathbb {E}\left[X_{n}\right]$ for all $n$, and such that $\operatorname {Cov}\left[X_{1}, X_{n}\right] \rightarrow 0$ as $n \rightarrow \infty$. Show that $n^{-1} \sum_{j = 1}^{n} X_{j} \xrightarrow {\text{ m.s. }} \mu$.

Deduce the strong law of large numbers from an appropriate ergodic theorem.

Let $\mathbb {Q}$ be a stationary measure on $\left(\mathbb {R}^{T}, \mathscr {B}^{T}\right)$ where $T = \left\{ 1,2, \ldots \right\}$. Show that $\mathbb {Q}$ is ergodic if and only if

for all $Y: \mathbb {R}^{T} \rightarrow \mathbb {R}$ for which $\mathbb {E}\left[Y\right]$ exists, where $Y_{i}: \mathbb {R}^{T} \rightarrow \mathbb {R}$ is given by $Y_{i}(\mathbf{x}) = Y\left(\tau^{i-1}(\mathbf{x})\right)$. As usual, $\tau$ is the natural shift operator on $\mathbb {R}^{T}$.

The stationary measure $\mathbb {Q}$ on $\left(\mathbb {R}^{T}, \mathscr {B}^{T}\right)$ is called strongly mixing if $\mathbb {Q}\left(A \cap \tau^{-n} B\right) \rightarrow \mathbb {Q}\left(A\right) \mathbb {Q}\left(B\right)$ as $n \rightarrow \infty$, for all $A, B \in \mathscr {B}^{T}$; as usual, $T = \left\{ 1,2, \ldots \right\}$ and $\tau$ is the shift operator on $\mathbb {R}^{T}$. Show that every strongly mixing measure is ergodic.

Let $(\Omega , \mathcal{F}, \mathbb {P})$ be a probability space, and let $T: \Omega \rightarrow \Omega$ be measurable and measure preserving (i.e. $\mathbb {P}\left(T^{-1} A\right) = \mathbb {P}\left(A\right)$ for all $A \in \mathcal{F}$). Let $X: \Omega \rightarrow \mathbb {R}$ be a random variable, and let $X_{i}$ be given by $X_{i}(\omega ) = X\left(T^{i-1}(\omega )\right)$. Show that

where $l$ is the $\sigma$-field of invariant events of $T$. If $T$ is ergodic (in that $\mathbb {P}\left(A\right)$ equals 0 or 1 whenever $A$ is invariant), prove that $\mathbb {E}\left[X \mid \ell \right] = \mathbb {E}\left[X\right]$ almost surely.

Consider the probability space $(\Omega , \mathcal{F}, \mathbb {P})$ where $\Omega = [0,1)$, $\mathcal{F}$ is the set of Borel subsets, and $\mathbb {P}$ is Lebesgue measure.

Let $g: \mathbb {R} \rightarrow \mathbb {R}$ be periodic with period $1$, and uniformly continuous and integrable over $[0,1]$. Define $Z_{n} = g(X+(n-1) \alpha ), n \geq 1$, where $X$ is uniform on $[0,1]$ and $\alpha$ is irrational. Show that, as $n \rightarrow \infty$,

Let $X = \left\{ X(t): t \geq 0\right\}$ be a non-decreasing random process such that:

(a) $X(0) = 0, X$ takes values in the non-negative integers,

(b) $X$ has stationary independent increments,

(c) the sample paths $\left\{ X(t, \omega ): t \geq 0\right\}$ have only jump discontinuities of unit magnitude. Show that $X$ is a Poisson process.

Let $X$ be a continuous-time process. Show that:

A Wiener process $W$ is called standard if $W(0) = 0$ and $W(1)$ has unit variance. Let $W$ be a standard Wiener process, and let $\alpha$ be a positive constant. Show that:

Let $W$ be a standard Wiener process. Show that the stochastic integrals

are well defined, and prove that $X(t) = W(t)$, and that $Y$ has autocovariance function $\operatorname {Cov}\left[Y(s), Y(t)\right] = \frac{1}{2}\left(e^{-\left|s-t\right|}-e^{-s-t}\right), s < t$.

Let $W$ be a standard Wiener process. Find the means of the following three processes, and the autocovariance functions in cases (b) and (c):

Let $W$ be a standard Wiener process. Find the conditional joint density function of $W\left(t_{2}\right)$ and $W\left(t_{3}\right)$ given that $W\left(t_{1}\right) = W\left(t_{4}\right) = 0$, where $t_{1} < t_{2} < t_{3} < t_{4}$.

Show that the conditional correlation of $W\left(t_{2}\right)$ and $W\left(t_{3}\right)$ is

Let $U_{1}, U_{2}, \ldots$ be independent random variables with the uniform distribution on $[0,1]$. Let $I_{j}(x)$ be the indicator function of the event $\left\{ U_{j} \leq x\right\}$, and define

The function $F_{n}$ is called the 'empirical distribution function' of the $U_{j}$.

An urn contains initially one red ball and one green ball. At later stages, a ball is picked from the urn uniformly at random, and is returned to the urn together with a fresh ball of the same colour. Assume the usual independence. Let $X_{k}$ be the indicator function of the event that the $k$ th ball picked is red.