Problems

Let $\ldots , X_{-1}, X_{0}, X_{1}, \ldots$ be uncorrelated random variables with zero means and unit variances, and define

where $\alpha$ and $\beta$ are constants satisfying $\left|\beta \right| < 1,\left|\beta -\alpha \right| < 1$. Find the best linear predictor of $Y_{n+1}$ given the entire past $Y_{n}, Y_{n-1}, \ldots$

Let $\left\{ Y_{k}:-\infty < k < \infty \right\}$ be a stationary sequence with variance $\sigma_{Y}^{2}$, and let

where $a_{0}, a_{1}, \ldots , a_{r}$ are constants. Show that $X$ has spectral density function

where $f_{Y}$ is the spectral density function of $Y, \sigma_{X}^{2} = \operatorname {Var}\left[X_{1}\right]$, and $G_{a}(z) = \sum_{k = 0}^{r} a_{k} z^{k}$. Calculate this spectral density explicitly in the case of 'exponential smoothing', when $r = \infty$, $a_{k} = \mu^{k}(1-\mu )$, and $0 < \mu < 1$.

Suppose that $\widehat{Y}_{n+1} = \alpha Y_{n}+\beta Y_{n-1}$ is the best linear predictor of $Y_{n+1}$ given the entire past $Y_{n}, Y_{n-1}, \ldots$ of the stationary sequence $\left\{ Y_{k}:-\infty < k < \infty \right\}$. Find the spectral density function of the sequence.

Meteorites fall at integer times $T_{1}, T_{2}, \ldots$ where $T_{n} = X_{1}+X_{2}+\cdots +X_{n}$. We assume that the $X_{i}$ are independent, $X_{2}, X_{3}, \ldots$ are identically distributed, and the distribution of $X_{1}$ is such that the probability that a meteorite falls at time $n$ is constant for all $n$. Let $Y_{n}$ be the indicator function of the event that a meteorite falls at time $n$. Show that $\left\{ Y_{n}\right\}$ is stationary and find its spectral density function in terms of the characteristic function of $X_{2}$.

Let $X = \left\{ X_{n}: n \geq 1\right\}$ be given by $X_{n} = \cos (n U)$ where $U$ is uniformly distributed on $[-\pi , \pi ]$. Show that $X$ is stationary but not strongly stationary. Find the autocorrelation function of $X$ and its spectral density function.

Let $Z_{1}, Z_{2}, \ldots$ be uncorrelated variables, each with zero mean and unit variance.

Show that the complex-valued stationary process $X = \left\{ X(t):-\infty < t < \infty \right\}$ has a spectral density function which is bounded and uniformly continuous whenever its autocorrelation function $\rho$ is continuous and satisfies $\int_{0}^{\infty }\left|\rho (t)\right| d t < \infty$.

Let $X = \left\{ X_{n}: n \geq 1\right\}$ be stationary with constant mean $\mu = \mathbb {E}\left[X_{n}\right]$ for all $n$, and such that $\operatorname {Cov}\left[X_{1}, X_{n}\right] \rightarrow 0$ as $n \rightarrow \infty$. Show that $n^{-1} \sum_{j = 1}^{n} X_{j} \xrightarrow {\text{ m.s. }} \mu$.

Deduce the strong law of large numbers from an appropriate ergodic theorem.

Let $\mathbb {Q}$ be a stationary measure on $\left(\mathbb {R}^{T}, \mathscr {B}^{T}\right)$ where $T = \left\{ 1,2, \ldots \right\}$. Show that $\mathbb {Q}$ is ergodic if and only if

for all $Y: \mathbb {R}^{T} \rightarrow \mathbb {R}$ for which $\mathbb {E}\left[Y\right]$ exists, where $Y_{i}: \mathbb {R}^{T} \rightarrow \mathbb {R}$ is given by $Y_{i}(\mathbf{x}) = Y\left(\tau^{i-1}(\mathbf{x})\right)$. As usual, $\tau$ is the natural shift operator on $\mathbb {R}^{T}$.

The stationary measure $\mathbb {Q}$ on $\left(\mathbb {R}^{T}, \mathscr {B}^{T}\right)$ is called strongly mixing if $\mathbb {Q}\left(A \cap \tau^{-n} B\right) \rightarrow \mathbb {Q}\left(A\right) \mathbb {Q}\left(B\right)$ as $n \rightarrow \infty$, for all $A, B \in \mathscr {B}^{T}$; as usual, $T = \left\{ 1,2, \ldots \right\}$ and $\tau$ is the shift operator on $\mathbb {R}^{T}$. Show that every strongly mixing measure is ergodic.

Let $(\Omega , \mathcal{F}, \mathbb {P})$ be a probability space, and let $T: \Omega \rightarrow \Omega$ be measurable and measure preserving (i.e. $\mathbb {P}\left(T^{-1} A\right) = \mathbb {P}\left(A\right)$ for all $A \in \mathcal{F}$). Let $X: \Omega \rightarrow \mathbb {R}$ be a random variable, and let $X_{i}$ be given by $X_{i}(\omega ) = X\left(T^{i-1}(\omega )\right)$. Show that

where $l$ is the $\sigma$-field of invariant events of $T$. If $T$ is ergodic (in that $\mathbb {P}\left(A\right)$ equals 0 or 1 whenever $A$ is invariant), prove that $\mathbb {E}\left[X \mid \ell \right] = \mathbb {E}\left[X\right]$ almost surely.

Consider the probability space $(\Omega , \mathcal{F}, \mathbb {P})$ where $\Omega = [0,1)$, $\mathcal{F}$ is the set of Borel subsets, and $\mathbb {P}$ is Lebesgue measure.

Let $g: \mathbb {R} \rightarrow \mathbb {R}$ be periodic with period $1$, and uniformly continuous and integrable over $[0,1]$. Define $Z_{n} = g(X+(n-1) \alpha ), n \geq 1$, where $X$ is uniform on $[0,1]$ and $\alpha$ is irrational. Show that, as $n \rightarrow \infty$,

Let $X = \left\{ X(t): t \geq 0\right\}$ be a non-decreasing random process such that:

(a) $X(0) = 0, X$ takes values in the non-negative integers,

(b) $X$ has stationary independent increments,

(c) the sample paths $\left\{ X(t, \omega ): t \geq 0\right\}$ have only jump discontinuities of unit magnitude. Show that $X$ is a Poisson process.

Let $X$ be a continuous-time process. Show that:

A Wiener process $W$ is called standard if $W(0) = 0$ and $W(1)$ has unit variance. Let $W$ be a standard Wiener process, and let $\alpha$ be a positive constant. Show that:

Let $W$ be a standard Wiener process. Show that the stochastic integrals

are well defined, and prove that $X(t) = W(t)$, and that $Y$ has autocovariance function $\operatorname {Cov}\left[Y(s), Y(t)\right] = \frac{1}{2}\left(e^{-\left|s-t\right|}-e^{-s-t}\right), s < t$.

Let $W$ be a standard Wiener process. Find the means of the following three processes, and the autocovariance functions in cases (b) and (c):

Let $W$ be a standard Wiener process. Find the conditional joint density function of $W\left(t_{2}\right)$ and $W\left(t_{3}\right)$ given that $W\left(t_{1}\right) = W\left(t_{4}\right) = 0$, where $t_{1} < t_{2} < t_{3} < t_{4}$.

Show that the conditional correlation of $W\left(t_{2}\right)$ and $W\left(t_{3}\right)$ is

Let $U_{1}, U_{2}, \ldots$ be independent random variables with the uniform distribution on $[0,1]$. Let $I_{j}(x)$ be the indicator function of the event $\left\{ U_{j} \leq x\right\}$, and define

The function $F_{n}$ is called the 'empirical distribution function' of the $U_{j}$.

An urn contains initially one red ball and one green ball. At later stages, a ball is picked from the urn uniformly at random, and is returned to the urn together with a fresh ball of the same colour. Assume the usual independence. Let $X_{k}$ be the indicator function of the event that the $k$ th ball picked is red.