Problems

Let $X_{n}$ have density function

With respect to which modes of convergence does $X_{n}$ converge as $n \rightarrow \infty$?

(i) Suppose that $X_{n} \xrightarrow {\text{ a.s. }} X$ and $Y_{n} \xrightarrow {\text{ a.s. }} Y$.

Let $g: \mathbb {R} \rightarrow \mathbb {R}$ be continuous. Show that $g\left(X_{n}\right) \xrightarrow {\mathrm{P}} g(X)$ if $X_{n} \xrightarrow {\mathrm{P}} X$.

Let $Y_{1}, Y_{2}, \ldots$ be independent identically distributed variables, each of which can take any value in $\left\{ 0,1, \ldots , 9\right\}$ with equal probability $\frac{1}{10}$. Let $X_{n} = \sum_{i = 1}^{n} Y_{i} 10^{-i}$. Show by the use of characteristic functions that $X_{n}$ converges in distribution to the uniform distribution on [0, 1]. Deduce that $X_{n} \xrightarrow {\text{ a.s. }} Y$ for some $Y$ which is uniformly distributed on $[0,1]$.

Let $N(t)$ be a Poisson process with constant intensity on $\mathbb {R}$.

Prove that $n^{-1} \sum_{i = 1}^{n} X_{i} \xrightarrow {\text{ a.s. }} 0$ whenever the $X_{i}$ are independent identically distributed variables with zero means and such that $\mathbb {E}\left[X_{1}^{4}\right] < \infty$.

Show that $X_{n} \xrightarrow {\text{ a.s. }} X$ whenever $\sum_{n} \mathbb {E}\left[\left|X_{n}-X\right|^{r}\right] < \infty$ for some $r > 0$.

Show that if $X_{n} \xrightarrow {\mathrm{D}} X$ then $a X_{n}+b \xrightarrow {\mathrm{D}} a X+b$ for any real $a$ and $b$.

Show that $X_{n} \xrightarrow {\mathrm{P}} 0$ if and only if

The sequence $\left\{ X_{n}\right\}$ is said to be mean-square Cauchy convergent if $\mathbb {E}\left[\left(X_{n}-X_{m}\right)^{2}\right] \rightarrow 0$ as $m, n \rightarrow \infty$. Show that $\left\{ X_{n}\right\}$ converges in mean square to some limit $X$ if and only if it is mean-square Cauchy convergent. Does the corresponding result hold for the other modes of convergence?

Suppose that $\left\{ X_{n}\right\}$ is a sequence of uncorrelated variables with zero means and uniformly bounded variances. Show that $n^{-1} \sum_{i = 1}^{n} X_{i} \xrightarrow {\text{ m.s. }} 0$.

Let $X_{1}, X_{2}, \ldots$ be independent identically distributed random variables with the common distribution function $F$, and suppose that $F(x) < 1$ for all $x$. Let $M_{n} = \max \left\{ X_{1}, X_{2}, \ldots , X_{n}\right\}$ and suppose that there exists a strictly increasing unbounded positive sequence $a_{1}, a_{2}, \ldots$ such that $\mathbb {P}\left(M_{n} / a_{n} \leq x\right) \rightarrow H(x)$ for some distribution function $H$. Let us assume that $H$ is continuous with $0 < H(1) < 1$; substantially weaker conditions suffice but introduce extra difficulties.

Let $X_{1}, X_{2}, \ldots , X_{n}$ be independent and identically distributed random variables with the Cauchy distribution. Show that $M_{n} = \max \left\{ X_{1}, X_{2}, \ldots , X_{n}\right\}$ is such that $\pi M_{n} / n$ converges in distribution, the limiting distribution function being given by $H(x) = e^{-1 / x}$ if $x \geq 0$.

Let $X_{1}, X_{2}, \ldots$ be independent and identically distributed random variables whose common characteristic function $\phi$ satisfies $\phi^{\prime }(0) = i \mu$. Show that $n^{-1} \sum_{j = 1}^{n} X_{j} \xrightarrow {\mathrm{P}} \mu$.

The total variation distance $d_{\mathrm{TV}}(X, Y)$ between two random variables $X$ and $Y$ is defined by

where the supremum is over all (measurable) functions $u: \mathbb {R} \rightarrow \mathbb {R}$ such that $\left\| u\right\|_{\infty } = \sup_{x}\left|u(x)\right|$ satisfies $\left\| u\right\|_{\infty } = 1$.

Let $g: \mathbb {R} \rightarrow \mathbb {R}$ be bounded and continuous. Show that

Let $X_{n}$ and $Y_{m}$ be independent random variables having the Poisson distribution with parameters $n$ and $m$, respectively. Show that

Let $X_{1}, X_{2}, \ldots$ be random variables satisfying $\operatorname {Var}\left[X_{n}\right] < c$ for all $n$ and some constant $c$. Show that the sequence obeys the weak law, in the sense that $n^{-1} \sum_{1}^{n}\left(X_{i}-\mathbb {E}\left[X_{i}\right]\right)$ converges in probability to $0$, if the correlation coefficients satisfy either of the following:

(i) $\rho \left(X_{i}, X_{j}\right) \leq 0$ for all $i \neq j$,

(ii) $\rho \left(X_{i}, X_{j}\right) \rightarrow 0$ as $\left|i-j\right| \rightarrow \infty$.

Let $X_{1}, X_{2}, \ldots$ be independent random variables with common density function

where $c$ is a constant. Show that the $X_{i}$ have no mean, but $n^{-1} \sum_{i = 1}^{n} X_{i} \xrightarrow {\mathrm{P}} 0$ as $n \rightarrow \infty$. Show that convergence does not take place almost surely.

Let $X_{n}$ be the Euclidean distance between two points chosen independently and uniformly from the $n$-dimensional unit cube. Show that $\mathbb {E}\left[X_{n}\right] / \sqrt{n} \rightarrow 1 / \sqrt{6}$ as $n \rightarrow \infty$.

Let $X_{1}, X_{2}, \ldots$ be independent random variables having the uniform distribution on $[-1,1]$. Show that

Let $X_{1}, X_{2}, \ldots$ be independent random variables, each $X_{k}$ having mass function given by

Show that $U_{n} = \sum_{1}^{n} X_{i}$ satisfies $U_{n} / \sqrt{n} \xrightarrow {\mathrm{D}} N(0,1)$ but $\operatorname {Var}\left[U_{n} / \sqrt{n}\right] \rightarrow 2$ as $n \rightarrow \infty$.

Let $X_{1}, X_{2}, \ldots$ be random variables, and let $N_{1}, N_{2}, \ldots$ be random variables taking values in the positive integers such that $N_{k} \xrightarrow {\mathrm{P}} \infty$ as $k \rightarrow \infty$. Show that:

(i) if $X_{n} \xrightarrow {\mathrm{D}} X$ and the $X_{n}$ are independent of the $N_{k}$, then $X_{N_{k}} \xrightarrow {\mathrm{D}} X$ as $k \rightarrow \infty$,

(ii) if $X_{n} \xrightarrow {\text{ a.s. }} X$ then $X_{N_{k}} \xrightarrow {\mathrm{P}} X$ as $k \rightarrow \infty$.

A bag contains red and green balls. A ball is drawn from the bag, its colour noted, and then it is returned to the bag together with a new ball of the same colour. Initially the bag contained one ball of each colour. If $R_{n}$ denotes the number of red balls in the bag after $n$ additions, show that $S_{n} = R_{n} /(n+2)$ is a martingale. Deduce that the ratio of red to green balls converges almost surely to some limit as $n \rightarrow \infty$.

Let $\left\{ X_{i}: i \geq 1\right\}$ be independent identically distributed random variables with zero mean and finite positive variance $\sigma^{2}$, and let $S_{n} = \sum_{1}^{n} X_{i}$. Suppose that the integer-valued random process $M(t)$ satisfies $t^{-1} M(t) \xrightarrow {\mathrm{P}} \theta$ as $t \rightarrow \infty$, where $\theta$ is a positive constant. Show that

You should not assume that the process $M$ is independent of the $X_{i}$.

Let $X_{1}, X_{2}, \ldots$ be independent random variables with zero means, and $S_{n} = X_{1}+X_{2}+\cdots +X_{n}$. Let $M_{n} = \max_{1 \leq k \leq n}\left|S_{k}\right|$ and show that $\mathbb {E}\left[S_{n}^{2} I_{A_{k}}\right] > c^{2} \mathbb {P}\left(A_{k}\right)$ where $A_{k} = \left\{ M_{k-1} \leq c < M_{k}\right\}$ and $c > 0$. Deduce Kolmogorov's inequality:

Let $X_{1}, X_{2}, \ldots$ be independent random variables with zero means, and let $S_{n} = X_{1}+X_{2}+ \cdots +X_{n}$. Using Kolmogorov's inequality or the martingale convergence theorem, show that:

(i) $\sum_{i = 1}^{\infty } X_{i}$ converges almost surely if $\sum_{k = 1}^{\infty } \mathbb {E}\left[X_{k}^{2}\right] < \infty$,

(ii) if there exists an increasing real sequence $(b_{n})$ such that $b_{n} \rightarrow \infty$, and satisfying the inequality $\sum_{k = 1}^{\infty } \mathbb {E}\left[X_{k}^{2}\right] / b_{k}^{2} < \infty$, then $b_{n}^{-1} \sum_{k = 1}^{\infty } X_{k} \xrightarrow {\text{ a.s. }} 0$ as $n \rightarrow \infty$.

The Markov chain $X_{0}, X_{1}, \ldots , X_{n}$ has initial distribution $f_{i} = \mathbb {P}\left(X_{0} = i\right)$ and transition matrix $\mathbf{P}$. The log-likelihood function $\lambda (\mathbf{P})$ is defined as $\lambda (\mathbf{P}) = \log \left(f_{X_{0}} p_{X_{0}, X_{1}} p_{X_{1}, X_{2}} \cdots p_{X_{n-1}, X_{n}}\right)$. Show that:

Ergodic theorem in discrete time. Let $X$ be an irreducible discrete-time Markov chain, and let $\mu_{i}$ be the mean recurrence time of state $i$. Let $V_{i}(n) = \sum_{r = 0}^{n-1} I_{\left\{ X_{r} = i\right\} }$ be the number of visits to $i$ up to $n-1$, and let $f$ be any bounded function on $S$. Show that:

Ergodic theorem in continuous time. Let $X$ be an irreducible recurrent continuous-time Markov chain with generator $\mathbf{G}$ and finite mean return times $m_{j}$.

Suppose that the sequences $\left\{ X_{n}: n \geq 1\right\}$ and $\left\{ Y_{n}: n \geq 1\right\}$ are tail equivalent, which is to say that $\sum_{n = 1}^{\infty } \mathbb {P}\left(X_{n} \neq Y_{n}\right) < \infty$. Show that:

Let $\left\{ X_{n}: n \geq 1\right\}$ be independent random variables. Show that $\sum_{n = 1}^{\infty } X_{n}$ converges a.s. if, for some $a > 0$, the following three series all converge:

(a) $\sum_{n} \mathbb {P}\left(\left|X_{n}\right| > a\right)$

(b) $\sum_{n} \operatorname {Var}\left[X_{n} I_{\left\{ \left|X_{n}\right| \leq a\right\} }\right]$

(c) $\sum_{n} \mathbb {E}\left[X_{n} I_{\left\{ \left|X_{n}\right| \leq a\right\} }\right]$.

Let $\left\{ X_{n}: n \geq 1\right\}$ be independent random variables with continuous common distribution function $F$. We call $X_{k}$ a record value for the sequence if $X_{k} > X_{r}$ for $1 \leq r < k$, and we write $I_{k}$ for the indicator function of the event that $X_{k}$ is a record value.

Let $\left\{ X_{n}: n \geq 1\right\}$ be a sequence of independent random variables with $\mathbb {P}\left(X_{n} = 1\right) = \mathbb {P}\left(X_{n} = -1\right) = \frac{1}{2}$. Does the series $\sum_{r = 1}^{n} X_{r} / r$ converge a.s. as $n \rightarrow \infty$?

Stirling's formula for the gamma function. Let $X$ have the gamma distribution $\Gamma (1, s)$. By considering the integral of the density function of $Y = (X-s) / \sqrt{s}$, show that $\Gamma (s) \sim \sqrt{2 \pi } s^{s-\frac{1}{2}} e^{-s}$ as $s \rightarrow \infty$. [Hint: You may find it useful that

if $u^{\prime }(x)$ is strictly positive and increasing.]

Let $c_{1}, c_{2}, \ldots$ be reals, let $X_{1}, X_{2}, \ldots$ be independent random variables with the mass function $f(1) = f(-1) = \frac{1}{2}$, and let $S_{n} = \sum_{r = 1}^{n} c_{r} X_{r}$. Write

Let $X$ and $Y$ be random variables with mean $0$, variance $1$, and correlation $\rho$. Show that, for $\epsilon > 0$,

where $x \vee y = \max \left\{ x, y\right\}$. [Hint: If $\left|t\right| \leq 1$, the function $g(x, y) = \left(x^{2}-2 t x y+y^{2}\right) /\left(\epsilon^{2}\left(1-t^{2}\right)\right)$ is non-negative, and moreover satisfies $g(x, y) \geq 1$ when $\left|x\right| \vee \left|y\right| \geq \epsilon$.]

Poisson tail, balls in bins. Let $X$ have the Poisson distribution with parameter 1.