Convergence of random variables

Let $r \geq 1$, and define $\left\| X\right\|_{r} = \mathbb {E}\left[\left|X^{r}\right|\right]^{1 / r}$. Show that:

Define $\langle X, Y\rangle = \mathbb {E}\left[X Y\right]$ for random variables $X$ and $Y$ having finite variance, and define $\left\| X\right\| = \sqrt{ }\langle X, X\rangle$. Show that:

Let $\epsilon > 0$. Let $g, h:[0,1] \rightarrow \mathbb {R}$, and define $d_{\epsilon }(g, h) = \int_{E} d x$ where $E = \left\{ u \in [0,1] : \left|g(u)-h(u)\right| > \epsilon \right\}$. Show that $d_{\epsilon }$ does not satisfy the triangle inequality.

For two distribution functions $F$ and $G$, let

Show that $d$ is a metric on the space of distribution functions.

Find random variables $X, X_{1}, X_{2}, \ldots$ such that $\mathbb {E}\left[\left|X_{n}-X\right|^{2}\right] \rightarrow 0$ as $n \rightarrow \infty$, but $\mathbb {E}\left[\left|X_{n}\right|\right] = \infty$ for all $n$.

Suppose $\left|X_{n}\right| \leq Z$ for all $n$, where $\mathbb {E}\left[Z\right] < \infty$. Prove that if $X_{n} \xrightarrow {\mathrm{P}} X$ then $X_{n} \xrightarrow {1} X$.

Show that convergence in distribution is equivalent to convergence with respect to the Lévy metric of Exercise (7.1.4).

Let $X_{1}, X_{2}, \ldots$ be random variables on the probability space $(\Omega , \mathcal{F}, \mathbb {P})$. Show that the set $A = \left\{ \omega \in \Omega \colon \text{the sequence } X_{n}(\omega ) \text{ converges}\right\}$ is an event (that is, lies in $\mathcal{F}$), and that there exists a random variable $X$ (that is, an $\mathcal{F}$-measurable function $X: \Omega \rightarrow \mathbb {R}$) such that $X_{n}(\omega ) \rightarrow X(\omega )$ for $\omega \in A$.

Let $\left\{ X_{n}\right\}$ be a sequence of random variables, and let $\left\{ c_{n}\right\}$ be a sequence of reals converging to the limit $c$. Show that the convergence of $X_{n}$ to $X$ entails the convergence of $c_{n} X_{n}$ to $c X$, for convergence

Let $\left\{ X_{n}\right\}$ be a sequence of independent random variables which converges in probability to the limit $X$. Show that $X$ is almost surely constant.

The sequence of discrete random variables $X_{n}$, with mass functions $f_{n}$, is said to converge in total variation to $X$ with mass function $f$ if

Suppose $X_{n} \rightarrow X$ in total variation, and $u: \mathbb {R} \rightarrow \mathbb {R}$ is bounded. Show that $\mathbb {E}\left[u\left(X_{n}\right)\right] \rightarrow \mathbb {E}\left[u(X)\right]$.

Let $\left\{ X_{r}: r \geq 1\right\}$ be independent Poisson variables with respective parameters $\left\{ \lambda_{r}: r \geq 1\right\}$. Show that $\sum_{r = 1}^{\infty } X_{r}$ converges or diverges almost surely according as $\sum_{r = 1}^{\infty } \lambda_{r}$ converges or diverges.

Let $X_{1}, X_{2}, \ldots$ be independent random variables, uniformly distributed on $\left\{ 1,2, \ldots , m\right\}$, and let $I_{m} = \min \left\{ r \geq 2: X_{r} = X_{S} \text{ for some } s < r\right\}$ be the earliest index of a coincidence of values. Show that $I_{m} / \sqrt{m}$ converges in distribution as $m \rightarrow \infty$, with as limit the Rayleigh distribution with density function $f(x) = x e^{-\frac{1}{2} x^{2}}, x > 0$.

The random variable $X$ has finite moments of all orders, and satisfies $\mathbb {P}\left(X > 0\right) = 1$, $\mathbb {P}\left(X > x\right) > 0$ for $x > 0$.

Show that the probability that infinitely many of the events $\left\{ A_{n}: n \geq 1\right\}$ occur satisfies $\mathbb {P}\left(A_{n} \text{ i.o.}\right) \geq \lim \sup_{n \rightarrow \infty } \mathbb {P}\left(A_{n}\right)$.

Let $\left\{ S_{n}: n \geq 0\right\}$ be a simple random walk which moves to the right with probability $p$ at each step, and suppose that $S_{0} = 0$. Write $X_{n} = S_{n}-S_{n-1}$.

Let $X_{1}, X_{2}, \ldots$ be independent identically distributed random variables. The event $A$, defined in terms of the $X_{n}$, is called exchangeable if $A$ is invariant under finite permutations of the coordinates, which is to say that its indicator function $I_{A}$ satisfies $I_{A}\left(X_{1}, X_{2}, \ldots , X_{n}, \ldots \right) = I_{A}\left(X_{i_{1}}, X_{i_{2}}, \ldots , X_{i_{n}}, X_{n+1}, \ldots \right)$ for all $n \geq 1$ and all permutations $\left(i_{1}, i_{2}, \ldots , i_{n}\right)$ of $(1,2, \ldots , n)$. Show that all exchangeable events $A$ are such that either $\mathbb {P}\left(A\right) = 0$ or $\mathbb {P}\left(A\right) = 1$.

Returning to the simple random walk $S$ of Exercise (7.3.3), show that $\left\{ S_{n} = 0 \text{ i.o.}\right\}$ is an exchangeable event with respect to the steps of the walk, and deduce from the Hewitt-Savage zero-one law that it has probability either 0 or 1.

Let $f:[0,1] \rightarrow \mathbb {R}$ be a continuous function, and let $S_{n}$ be a random variable having the binomial distribution with parameters $n$ and $x$. Using the formula $\mathbb {E}\left[Z\right] = \mathbb {E}\left[Z I_{A}\right]+\mathbb {E}\left[Z I_{A^{\complement }}\right]$ with $Z = f(x)-f\left(n^{-1} S_{n}\right)$ and $A = \left\{ \left|n^{-1} S_{n}-x\right| > \delta \right\}$, show that

You have proved Weierstrass's approximation theorem, which states that every continuous function on $[0,1]$ may be approximated by a polynomial uniformly over the interval.

A sequence $X_{1}, X_{2}, \ldots$ of random variables is said to be completely convergent to $X$ if

Show that, for sequences of independent variables, complete convergence is equivalent to a.s. convergence. Find a sequence of (dependent) random variables which converges a.s. but not completely.

Let $X_{1}, X_{2}, \ldots$ be independent identically distributed random variables with common mean $\mu$ and finite variance. Show that

Let $\left\{ X_{n}: n \geq 1\right\}$ be independent and exponentially distributed with parameter 1. Show that

Let $\left\{ X_{n}: n \geq 1\right\}$ be independent $N(0,1)$ random variables. Show that:

Construct an example to show that the convergence in distribution of $X_{n}$ to $X$ does not imply the convergence of the unique medians of the sequence $X_{n}$.

(i) Let $\left\{ X_{r}: r \geq 1\right\}$ be independent, non-negative and identically distributed with infinite mean. Show that $\lim_{\sup_{r \rightarrow \infty }} X_{r} / r = \infty$ almost surely.

(ii) Let $\left\{ X_{r}\right\}$ be a stationary Markov chain on the positive integers with transition probabilities

Let $\left\{ X_{r}: 1 \leq r \leq n\right\}$ be independent and identically distributed with mean $\mu$ and finite variance $\sigma^{2}$. Let $\overline{X} = n^{-1} \sum_{r = 1}^{n} X_{r}$. Show that

converges in distribution to the $N(0,1)$ distribution as $n \rightarrow \infty$.

For a random variable $X$ with mean $0$, variance $\sigma^{2}$, and $\mathbb {E}\left[X^{4}\right] < \infty$, show that

Let $X_{2}, X_{3}, \ldots$ be independent random variables such that

Show that this sequence obeys the weak law but not the strong law, in the sense that $n^{-1} \sum_{1}^{n} X_{i}$ converges to 0 in probability but not almost surely.

Construct a sequence $\left\{ X_{r}: r \geq 1\right\}$ of independent random variables with zero mean such that $n^{-1} \sum_{r = 1}^{n} X_{r} \rightarrow -\infty$ almost surely, as $n \rightarrow \infty$.

Let $N$ be a spatial Poisson process with constant intensity $\lambda$ in $\mathbb {R}^{d}$, where $d \geq 2$. Let $S$ be the ball of radius $r$ centred at zero. Show that $N(S) /\left|S\right| \rightarrow \lambda$ almost surely as $r \rightarrow \infty$, where $\left|S\right|$ is the volume of the ball.

In each of a sequence of independent bets, a gambler either wins $30 \%$, or loses $25 \%$ of her current fortune, each with probability $\frac{1}{2}$. Denoting her fortune after $n$ bets by $F_{n}$, show that $\mathbb {E}\left[F_{n}\right] \rightarrow \infty$ as $n \rightarrow \infty$, while $F_{n} \rightarrow 0$ almost surely.

Let $S_{n} = X_{1}+X_{2}+\cdots +X_{n}$ be the sum of independent, identically distributed random variables. Let $\delta , \epsilon > 0$, and define the truncated variables $Y_{j} = X_{j} I_{\left\{ \left|X_{j}\right| \leq \delta n\right\} }$. Let $A = \left\{ X_{j} = Y_{j} \text{ for } j = 1,2, \ldots , n\right\}$, and $B = \left\{ \left|S_{n}-n \mathbb {E}\left[Y_{1}\right]\right| \geq \epsilon n\right\}$. Prove that:

The interval $[0,1]$ is partitioned into $n$ disjoint sub-intervals with lengths $p_{1}, p_{2}, \ldots$, $p_{n}$, and the entropy of this partition is defined to be

Let $X_{1}, X_{2}, \ldots$ be independent random variables having the uniform distribution on $[0,1]$, and let $Z_{m}(i)$ be the number of the $X_{1}, X_{2}, \ldots , X_{m}$ which lie in the $i$ th interval of the partition above. Show that

satisfies $m^{-1} \log R_{m} \rightarrow -h$ almost surely as $m \rightarrow \infty$.

Catastrophes occur at the times $T_{1}, T_{2}, \ldots$ where $T_{i} = X_{1}+X_{2}+\cdots +X_{i}$ and the $X_{i}$ are independent identically distributed positive random variables. Let $N(t) = \max \left\{ n: T_{n} \leq t\right\}$ be the number of catastrophes which have occurred by time $t$. Prove that if $\mathbb {E}\left[X_{1}\right] < \infty$ then $N(t) \rightarrow \infty$ and $N(t) / t \rightarrow 1 / \mathbb {E}\left[X_{1}\right]$ as $t \rightarrow \infty$, almost surely.

Let $X_{1}, X_{2}, \ldots$ be independent identically distributed random variables taking values in the integers $\mathbb {Z}$ and having a finite mean. Show that the Markov chain $S = \left\{ S_{n}\right\}$ given by $S_{n} = \sum_{1}^{n} X_{i}$ is transient if $\mathbb {E}\left[X_{1}\right] \neq 0$.

A function $\phi (x)$ is said to belong to the 'upper class' if, in the notation of this section, $\mathbb {P}\left(S_{n} > \phi (n) \sqrt{n} \text{ i.o.}\right) = 0$. A consequence of the law of the iterated logarithm is that $\sqrt{\alpha \log \log x}$ is in the upper class for all $\alpha > 2$. Use the first Borel-Cantelli lemma to prove the much weaker fact that $\phi (x) = \sqrt{\alpha \log x}$ is in the upper class for all $\alpha > 2$, in the special case when the $X_{i}$ are independent $N(0,1)$ variables.

Let $X_{1}, X_{2}, \ldots$ be random variables such that the partial sums $S_{n} = X_{1}+X_{2}+\cdots +X_{n}$ determine a martingale. Show that $\mathbb {E}\left[X_{i} X_{j}\right] = 0$ if $i \neq j$.

Let $Z_{n}$ be the size of the $n$th generation of a branching process with immigration, in which the family sizes have mean $\mu ( \neq 1)$ and the mean number of immigrants in each generation is $m$. Suppose that $\mathbb {E}\left[Z_{0}\right] < \infty$, and show that

is a martingale with respect to a suitable sequence of random variables.

Let $X_{0}, X_{1}, X_{2}, \ldots$ be a sequence of random variables with finite means and satisfying $\mathbb {E}\left[X_{n+1} \mid X_{0}, X_{1}, \ldots , X_{n}\right] = a X_{n}+b X_{n-1}$ for $n \geq 1$, where $0 < a, b < 1$ and $a+b = 1$. Find a value of $\alpha$ for which $S_{n} = \alpha X_{n}+X_{n-1}, n \geq 1$, defines a martingale with respect to the sequence $X$.

Let $X_{n}$ be the net profit to the gambler of betting a unit stake on the $n$th play in a casino; the $X_{n}$ may be dependent, but the game is fair in the sense that $\mathbb {E}\left[X_{n+1} \mid X_{1}, X_{2}, \ldots , X_{n}\right] = 0$ for all $n$. The gambler stakes $Y$ on the first play, and thereafter stakes $f_{n}\left(X_{1}, X_{2}, \ldots , X_{n}\right)$ on the $(n+1)$ th play, where $f_{1}, f_{2}, \ldots$ are given functions. Show that her profit after $n$ plays is

where $f_{0} = Y$. Show further that the sequence $S = \left\{ S_{n}\right\}$ satisfies the martingale condition $\mathbb {E}\left[S_{n+1} \mid X_{1}, X_{2}, \ldots , X_{n}\right] = S_{n}, n \geq 1$, if $Y$ is assumed to be known throughout.

A run in a random permutation $\left(\pi_{1}, \pi_{2}, \ldots , \pi_{n}\right)$ of $(1,2, \ldots , n)$ is a subsequence satisfying $\pi_{r-1} > \pi_{r} < \pi_{r+1} < \cdots < \pi_{s} > \pi_{s+1}$. We set $\pi_{0} = n+1$ and $\pi_{n+1} = 0$ by convention. Let $R_{n}$ be the number of runs. Show that $M_{n} = n R_{n}-\frac{1}{2} n(n+1)$ is a martingale. Find $\mathbb {E}\left[R_{n}\right]$ and $\mathbb {E}\left[R_{n}^{2}\right]$.

Let $X_{1}, X_{2}, \ldots$ be independent random variables with zero means and finite variances, and let $S_{n} = X_{1}+X_{2}+\cdots +X_{n}$. Use the Doob-Kolmogorov inequality to show that

Let $X_{1}, X_{2}, \ldots$ be independent random variables such that $\sum_{n} n^{-2} \operatorname {Var}\left[X_{n}\right] < \infty$. Use Kolmogorov's inequality to prove that

for some finite random variable $Y$, and deduce that

(You may find Kronecker's lemma to be useful: if $\left(a_{n}\right)$ and $\left(b_{n}\right)$ are real sequences with $b_{n} \uparrow \infty$ and $\sum_{i} a_{i} / b_{i} < \infty$, then $b_{n}^{-1} \sum_{i = 1}^{n} a_{i} \rightarrow 0$ as $n \rightarrow \infty$.)

Let $S$ be a martingale with respect to $X$, such that $\mathbb {E}\left[S_{n}^{2}\right] < K < \infty$ for some $K \in \mathbb {R}$. Suppose that $\operatorname {Var}\left[S_{n}\right] \rightarrow 0$ as $n \rightarrow \infty$, and prove that $S = \lim_{n \rightarrow \infty } S_{n}$ exists and is constant almost surely.

Let $Y$ be uniformly distributed on $[-1,1]$ and let $X = Y^{2}$.

Let $X_{1}, X_{2}, \ldots , X_{n}$ be random variables with zero means and covariance matrix $\mathbf{V} = \left(v_{i j}\right)$, and let $Y$ have finite second moment. Find the linear function $h$ of the $X_{i}$ which minimizes the mean squared error $\mathbb {E}\left[\left(Y-h\left(X_{1}, \ldots , X_{n}\right)\right)^{2}\right]$.

Verify the following properties of conditional expectation. You may assume that the relevant expectations exist.

Let $X$ and $Y$ have joint mass function $f(x, y) = \left\{ x(x+1)\right\}^{-1}$ for $x = y = 1,2, \ldots$. Show that $\mathbb {E}\left[Y \mid X\right] < \infty$ while $\mathbb {E}\left[Y\right] = \infty$

Let $(\Omega , \mathcal{F}, \mathbb {P})$ be a probability space and let $\mathcal{G}$ be a sub-$\sigma$-field of $\mathcal{F}$. Let $H$ be the space of $\mathcal{G}$-measurable random variables with finite second moment.

For possibly dependent random variables $X$ and $Y$, define the maximal correlation coefficient $m(X, Y) = \sup \rho (f(X), g(Y))$, where $\rho$ denotes ordinary correlation, and the supremum is over all functions $f$ and $g$ such that $f(X)$ and $g(Y)$ have finite non-zero variances. Show that: