Problems

A die is thrown ten times. What is the probability that the sum of the scores is 27?

A coin is tossed repeatedly, heads appearing with probability $p$ on each toss.

A coin is tossed repeatedly, heads occurring on each toss with probability $p$. Find the probability generating function of the number $T$ of tosses before a run of $n$ heads has appeared for the first time.

Find the generating function of the negative binomial mass function

where $0 < p < 1$ and $r$ is a positive integer. Deduce the mean and variance.

For the simple random walk, show that the probability $p_{0}(2 n)$ that the particle returns to the origin at the $(2 n)$ th step satisfies $p_{0}(2 n) \sim (4 p q)^{n} / \sqrt{\pi n}$, and use this to prove that the walk is recurrent if and only if $p = \frac{1}{2}$. You will need Stirling's formula: $n!\sim n^{n+\frac{1}{2}} e^{-n} \sqrt{2 \pi }$.

A symmetric random walk in two dimensions is defined to be a sequence of points $\left\{ \left(X_{n}, Y_{n}\right) : n \geq 0\right\}$ which evolves in the following way: if $\left(X_{n}, Y_{n}\right) = (x, y)$ then $\left(X_{n+1}, Y_{n+1}\right)$ is one of the four points $(x \pm 1, y),(x, y \pm 1)$, each being picked with equal probability $\frac{1}{4}$. If $\left(X_{0}, Y_{0}\right) = (0,0)$ :

Consider the one-dimensional random walk $\left\{ S_{n}\right\}$ given by

where $0 < p < 1$.

Let $X$ and $Y$ be independent variables taking values in the positive integers such that

for some $p$ and all $0 \leq k \leq n$. Show that $X$ and $Y$ have Poisson distributions.

In a branching process whose family sizes have mean $\mu$ and variance $\sigma^{2}$, find the variance of $Z_{n}$, the size of the $n$th generation, given that $Z_{0} = 1$.

A group $\left\{ A_{1}, A_{2}, \ldots , A_{r}\right\}$ of $r( > 2)$ people play the following game. $A_{1}$ and $A_{2}$ wager on the toss of a fair coin. The loser puts $£ 1$ in the pool, the winner goes on to play $A_{3}$. In the next wager, the loser puts $£ 1$ in the pool, the winner goes on to play $A_{4}$, and so on. The winner of the $(r-1)$ th wager goes on to play $A_{1}$, and the cycle recommences. The first person to beat all the others in sequence takes the pool.

A branching process has a single progenitor.

Show that the number $Z_{n}$ of individuals in the $n$th generation of a branching process satisfies $\mathbb {P}\left(Z_{n} > N \mid Z_{m} = 0\right) \leq G_{m}(0)^{N}$ for $n < m$.

The distribution of a random variable $X$ is called infinitely divisible if, for all positive integers $n$, there exists a sequence $Y_{1}^{(n)}, Y_{2}^{(n)}, \ldots , Y_{n}^{(n)}$ of independent identically distributed random variables such that $X$ and $Y_{1}^{(n)}+Y_{2}^{(n)}+\cdots +Y_{n}^{(n)}$ have the same distribution.

Let $X_{1}, X_{2}, \ldots$ be independent variables each taking the values 0 or 1 with probabilities $1-p$ and $p$, where $0 < p < 1$. Let $N$ be a random variable taking values in the positive integers, independent of the $X_{i}$, and write $S = X_{1}+X_{2}+\cdots +X_{N}$. Write down the conditional generating function of $N$ given that $S = N$, in terms of the probability generating function $G$ of $N$. Show that $N$ has a Poisson distribution if and only if $\mathbb {E}\left[x^{N}\right]^{p} = \mathbb {E}\left[x^{N} \mid S = N\right]$ for all $p$ and $x$.

If $X$ and $Y$ have joint probability generating function

find the marginal mass functions of $X$ and $Y$, and the mass function of $X+Y$. Find also the conditional probability generating function $G_{X \mid Y}(s \mid y) = \mathbb {E}\left[s^{X} \mid Y = y\right]$ of $X$ given that $Y = y$. The pair $X, Y$ is said to have the bivariate negative binomial distribution.

If $X$ and $Y$ have joint probability generating function

find the marginal distributions of $X, Y$, and the distribution of $X+Y$, showing that $X$ and $Y$ have the Poisson distribution, but that $X+Y$ does not unless $\gamma = 0$.

Define

for $a, b > 0$. Show that

Let $X, Y, Z$ be independent $N(0,1)$ variables. Use characteristic functions and moment generating functions (Laplace transforms) to find the distributions of

Let $X$ have density function $f$ and characteristic function $\phi$, and suppose that $\int_{-\infty }^{\infty }\left|\phi (t)\right| d t < \infty$. Deduce that

Consider a branching process whose family sizes have the geometric mass function $f(k) = q p^{k}, k \geq 0$, where $\mu = p / q > 1$. Let $Z_{n}$ be the size of the $n$th generation, and assume $Z_{0} = 1$. Show that the conditional distribution of $Z_{n} / \mu^{n}$, given that $Z_{n} > 0$, converges as $n \rightarrow \infty$ to the exponential distribution with parameter $1-\mu^{-1}$.

A random variable $X$ is called symmetric if $X$ and $-X$ are identically distributed. Show that $X$ is symmetric if and only if the imaginary part of its characteristic function is identically zero.

Let $X$ and $Y$ be independent identically distributed variables with means 0 and variances 1. Let $\phi (t)$ be their common characteristic function.

Let $X$ and $Y$ be independent random variables each having the Cauchy density function $f(x) = \left\{ \pi \left(1+x^{2}\right)\right\}^{-1}$, and let $Z = \frac{1}{2}(X+Y)$.

Let $X_{1}, X_{2}, \ldots , X_{n}$ be independent variables with characteristic functions $\phi_{1}, \phi_{2}, \ldots , \phi_{n}$. Describe random variables which have the following characteristic functions:

Which of the following are characteristic functions:

Show that the characteristic function $\phi$ of a random variable $X$ satisfies $\left|1-\phi (t)\right| \leq \mathbb {E}\left[\left|t X\right|\right]$.

Suppose $X$ and $Y$ have joint characteristic function $\phi (s, t)$. Show that, subject to the appropriate conditions of differentiability,

for any positive integers $m$ and $n$.

If $X$ has distribution function $F$ and characteristic function $\phi$, show that for $t > 0$

Let $X_{1}, X_{2}, \ldots$ be independent variables which are uniformly distributed on [0,1]. Let $M_{n} = \max \left\{ X_{1}, X_{2}, \ldots , X_{n}\right\}$ and show that $n\left(1-M_{n}\right) \xrightarrow {\mathrm{D}} X$ where $X$ is exponentially distributed with parameter 1. You need not use characteristic functions.

If $X$ is either

Recall that you regularly buy quantities of some ineffably dull commodity. To attract your attention, the manufacturers add to each packet a small object which is also dull, and in addition useless, but there are $n$ different types. Assume that each packet is equally likely to contain any one of the different types, as usual. Let $T_{n}$ be the number of packets bought before you acquire a complete set of $n$ objects. Show that $n^{-1}\left(T_{n}-n \log n\right) \xrightarrow {\mathrm{D}} T$, where $T$ is a random variable with distribution function $\mathbb {P}\left(T \leq x\right) = \exp \left(-e^{-x}\right),-\infty < x < \infty$.

Find a sequence $\left(\phi_{n}\right)$ of characteristic functions with the property that the limit given by $\phi (t) = \lim_{n \rightarrow \infty } \phi_{n}(t)$ exists for all $t$, but such that $\phi$ is not itself a characteristic function.

A biased coin is tossed $N$ times, where $N$ is a random variable which is Poisson distributed with parameter $\lambda$.

A binary tree is a tree (as in the section on branching processes) in which each node has exactly two descendants. Suppose that each node of the tree is coloured black with probability $p$, and white otherwise, independently of all other nodes. For any path $\pi$ containing $n$ nodes beginning at the root of the tree, let $B(\pi )$ be the number of black nodes in $\pi$, and let $X_{n}(k)$ be the number of such paths $\pi$ for which $B(\pi ) \geq k$. Show that there exists $\beta_{c}$ such that

and show how to determine the value $\beta_{c}$. Prove that

Use the continuity theorem (5.9.5) to show that, as $n \rightarrow \infty$,

Let $X_{1}, X_{2}, \ldots$ be independent random variables with zero means and such that $\mathbb {E}\left[\left|X_{j}^{3}\right|\right] < \infty$ for all $j$. Show that $S_{n} = X_{1}+X_{2}+\cdots +X_{n}$ satisfies $S_{n} / \sqrt{\operatorname {Var}\left[S_{n}\right]} \xrightarrow {\mathrm{D}} N(0,1)$ as $n \rightarrow \infty$ if

The following steps may be useful. Let $\sigma_{j}^{2} = \operatorname {Var}\left[X_{j}\right], \sigma (n)^{2} = \operatorname {Var}\left[S_{n}\right], \rho_{j} = \mathbb {E}\left[\left|X_{j}^{3}\right|\right]$, and $\phi_{j}$ and $\psi_{n}$ be the characteristic functions of $X_{j}$ and $S_{n} / \sigma (n)$ respectively.

(i) Use Taylor’s theorem to show that $\left|\phi_{j}(t)-1\right| \leq 2 t^{2} \sigma_{j}^{2}$ and $\left|\phi_{j}(t)-1+\frac{1}{2} \sigma_{j}^{2} t^{2}\right| \leq \left|t\right|^{3} \rho_{j}$ for $j \geq 1$.

(ii) Show that $\left|\log (1+z)-z\right| \leq \left|z\right|^{2}$ if $\left|z\right| \leq \frac{1}{2}$, where the logarithm has its principal value.

(iii) Show that $\sigma_{j}^{3} \leq \rho_{j}$, and deduce from the hypothesis that $\max_{1 \leq j \leq n} \sigma_{j} / \sigma (n) \rightarrow 0$ as $n \rightarrow \infty$, implying that $\max_{1 \leq j \leq n}\left|\phi_{j}(t / \sigma (n))-1\right| \rightarrow 0$.

(iv) Deduce an upper bound for $\left|\log \phi_{j}(t / \sigma (n))-\frac{1}{2} t^{2} \sigma_{j}^{2} / \sigma (n)^{2}\right|$, and sum to obtain that $\log \psi_{n}(t) \rightarrow -\frac{1}{2} t^{2}$.

Let $X_{1}, X_{2}, \ldots$ be independent variables each taking values $+1$ or $-1$ with probabilities $\frac{1}{2}$ and $\frac{1}{2}$. Show that

Let $X_{1}, X_{2}, \ldots , X_{n}$ be independent $N\left(\mu , \sigma^{2}\right)$ random variables. Define $\bar{X} = n^{-1} \sum_{1}^{n} X_{i}$ and $Z_{i} = X_{i}-\bar{X}$. Find the joint characteristic function of $\bar{X}, Z_{1}, Z_{2}, \ldots , Z_{n}$, and hence prove that $\bar{X}$ and $S^{2} = (n-1)^{-1} \sum_{1}^{n}\left(X_{i}-\bar{X}\right)^{2}$ are independent.

Let $X$ be $N(0,1)$, and let $Y = e^{X} ; Y$ is said to have the log-normal distribution. Show that the density function of $Y$ is

For $\left|a\right| \leq 1$, define $f_{a}(x) = \left\{ 1+a \sin (2 \pi \log x)\right\} f(x)$. Show that $f_{a}$ is a density function with finite moments of all (positive) orders, none of which depends on the value of $a$. The family $\left\{ f_{a}:\left|a\right| \leq 1\right\}$ contains density functions which are not specified by their moments.

Consider a random walk whose steps are independent and identically distributed integer-valued random variables with non-zero mean. Prove that the walk is transient.

Let $\left\{ X_{r}: r \geq 1\right\}$ be the integer-valued identically distributed intervals between the times of a recurrent event process. Let $L$ be the earliest time by which there has been an interval of length $a$ containing no occurrence time. Show that, for integral $a$,

A biased coin shows heads with probability $p( = 1-q)$. It is flipped repeatedly until the first time $W_{n}$ by which it has shown $n$ consecutive heads. Let $\mathbb {E}\left[s^{W_{n}}\right] = G_{n}(s)$. Show that $G_{n} = p s G_{n-1} /\left(1-q s G_{n-1}\right)$, and deduce that

In $n$ flips of a biased coin which shows heads with probability $p( = 1-q)$, let $L_{n}$ be the length of the longest run of heads. Show that, for $r \geq 1$,

The random process $\left\{ X_{n}: n \geq 1\right\}$ decays geometrically fast in that, in the absence of external input, $X_{n+1} = \frac{1}{2} X_{n}$. However, at any time $n$ the process is also increased by $Y_{n}$ with probability $\frac{1}{2}$, where $\left\{ Y_{n}: n \geq 1\right\}$ is a sequence of independent exponential random variables with parameter $\lambda$. Find the limiting distribution of $X_{n}$ as $n \rightarrow \infty$.

Let $G(s) = \mathbb {E}\left[s^{X}\right]$ where $X \geq 0$. Show that $\mathbb {E}\left[(X+1)^{-1}\right] = \int_{0}^{1} G(s) d s$, and evaluate this when $X$ is

Find the density function of $\sum_{r = 1}^{N} X_{r}$, where $\left\{ X_{r}: r \geq 1\right\}$ are independent and exponentially distributed with parameter $\lambda$, and $N$ is geometric with parameter $p$ and independent of the $X_{r}$.

Let $X$ have finite non-zero variance and characteristic function $\phi (t)$. Show that

is a characteristic function, and find the corresponding distribution.

Let $X$ and $Y$ have joint density function

Show that $\phi_{X}(t) \phi_{Y}(t) = \phi_{X+Y}(t)$, and that $X$ and $Y$ are dependent. Find the probability density function of $X+Y$.

Let $X_{1}, X_{2}, \ldots$ be independent and uniformly distributed on $(0,1)$, and let $m(x) = \mathbb {E}\left[N\right]$ where $N = \min \left\{ n: \sum_{r = 1}^{n} X_{r} > x\right\}$ for $x > 0$. Show that $m^{\prime }(x) = m(x)-m(x-1)$, and deduce that the Laplace transform $m^{*}(s) = \int_{0}^{\infty } m(x) e^{-s x} d x$ is given by $m^{*}(s) = 1 /\left(e^{-s}+s-1\right)$ for $s \neq 0$. Hence prove that

Let $S_{n} = \sum_{r = 1}^{n} X_{r}$ be a partial sum of a sequence of independent random variables with the uniform distribution on $(0,1)$. For $x \in \mathbb {R}$, let $\left\{ x\right\}$ denote the nearest integer to $x$, and write $R_{n} = \sum_{r = 1}^{n}\left\{ X_{r}\right\}$.