Miscellaneous Complements and Problems

In the study of sequences of events $A_{1}, A_{2}, \ldots$ with $p_{k} = \mathbb {P}\left(A_{k}\right)$, a significant role is played by the Borel--Cantelli lemma:

Suppose $\left\{ X_{n}, n \geq 1\right\}$ is a sequence of independent random variables, and for any $c > 0$, consider the truncated random variables

Then the series

converges with probability 1 if and only if the following three series converge:

Otherwise, $S$ converges with probability 0.

This law concerns the frequency of appearance of large values of the standardized number of successes,

in an infinite sequence of Bernoulli trials. According to the De Moivre--Laplace Central Limit Theorem (CLT), we have

where $\sim$ means that the ratio of the two sides tends to one.

Therefore, for a particular $n$, large values ($>4$) of $S_{n}^{*}$ are improbable, but, obviously, for large $n$, it is possible for $S_{n}^{*}$ to exceed, sooner or later, any large value. How soon this may occur is given by the Law of the Iterated Logarithm (Khintchine; see, e.g., Feller, 1957). With probability 1

i.e., for $\lambda > 1$, only a finite number of the events

occurs with probability 1, while for $\lambda < 1$, (2) holds for infinitely many $n$ with probability 1.

For reasons of symmetry (of the distribution of $S_{n}^{*}$), (1) implies

The behavior of $S_{n}$ is further illustrated by the following:

Let $\left\{ X_{n}, n \geq 1\right\}$ be a sequence of independent random variables and $A$ an event independent of (any event defined in terms of) $X_{1}, \ldots , X_{k}$ for every $k$. Then, either $\mathbb {P}\left(A\right) = 0$ or $\mathbb {P}\left(A\right) = 1$.

For large $x$ ($x \rightarrow \infty$), the approximation

is valid; more precisely, for every $x > 0$, the double inequality

is valid. Moreover, for every constant $a > 0$ show that, as $x \rightarrow \infty$,

Consider a sequence $\left\{ A_{n}, n \geq 1\right\}$ of independent events. Then a finite or infinite number of events $A_{k}$ occur with probability 1, according to whether the series $\sum_{n} \mathbb {P}\left(A_{n}\right)$ converges or diverges, respectively (see the above Borel--Cantelli Lemma). Show this by using the Three-Series Theorem.

Let $\left\{ X_{n}, n \geq 1\right\}$ be a sequence of random variables with $\mu_{n} = \mathbb {E}\left[X_{n}\right] < \infty$ and $X$ a random variable with $\operatorname {Var}\left[X\right] < \infty$. If, for each $k$, $X_{1}, \ldots , X_{k}$ and $X-(X_{1}+\cdots +X_{k})$ are independent, prove that $\operatorname {Var}\left[X_{k}\right] < \infty$ for every $k$, and that the series

converges with probability 1 (almost everywhere or almost surely).

In a sequence of (independent) Bernoulli trials with probability of success $p$, the event $A_{k}$ is realized if $k$ consecutive successes appear between the $2^{k}$th and $2^{k+1}$th trial. Prove that if $p \geq 1/2$, then an infinite number of the events $A_{k}$ occur with probability 1, whereas for $p < 1/2$ with probability 1 a finite number of the events $A_{k}$ occur.

A continuous function $\varphi (t)$ with $\varphi (0) = 1$ is a characteristic function if and only if it is nonnegative definite, i.e., if for each $n \geq 1$ and for each $n$-tuple of real numbers $t_{1}, \ldots , t_{n}$ and complex numbers $Z_{1}, \ldots , Z_{n}$ it satisfies the

Prove that $(*)$ is necessary.

Prove that the functions,

If $\varphi (t)$ is a characteristic function, prove that $g(t) = e^{\varphi (t)-1}$ is also a characteristic function.

Show that, for a real characteristic function $\varphi (t)$, the following inequalities are valid:

Let $X$ be the number of Bernoulli trials required until we have observed $r$ successive successes. Find the generating function of $X$ and $\mathbb {E}\left[X\right]$.

Let $M_{n}$ be the maximum number of consecutive successes (the maximum length of a run) observed in $n$ Bernoulli trials. If

show that

where $p_{n,r}$ was defined in the preceding problem (Problem 5.13, the number of Bernoulli trials required until $r$ successive successes are observed), and therefore, the generating function of $p_{n,r}$ is

Also, show that

Find the distributions with characteristic functions,

Find the discrete distributions with generating functions,

Using the Kolmogorov inequality: for independent $X_{1}, \ldots , X_{n}$, let $S_{k} = X_{1}+\cdots +X_{k}$ with $\mathbb {E}\left[S_{k}\right] = m_{k}$, $\operatorname {Var}\left[S_{k}\right] = s_{k}^{2}$ ($k = 1, 2, \ldots$); then for every $\varepsilon > 0$,

show that if the series

then the SLLN holds, i.e.,

The probability that, in each play of a game, the player receives $2^{k}$ dollars is

and that he receives 0 dollars is $p_{0} = 1-(p_{1}+p_{2}+\cdots )$. So, the expected (average) gain in each game is

If, in each game, the player pays a 1-dollar fee, the net profit of the player after $n$ games is equal to

i.e., the game is fair ($X_{k}$ is a random variable with distribution (1)). For every $\varepsilon > 0$, however, the probability that, in $n$ games, the player will lose more than $(1-\varepsilon ) n/\log_{2} n$ dollars, approaches 1, i.e., it can be proved (truncating the random variables $X_{k}$, see Feller (1957) or Gnedenko (1962)) that

In a sequence $\left\{ X_{n}, n \geq 1\right\}$ of Bernoulli random variables, suppose that

Find $\mathbb {E}\left[Z_{n}\right]$ and $\operatorname {Var}\left[Z_{n}\right]$ where $Z_{n} = \sum_{i=1}^{n} Y_{i}$.

We randomly choose two numbers in $(0,1)$. What is the probability $p$ that their sum be smaller than 1 and their product smaller than 3/16?

A bus on line $A$ arrives at a bus station every 4 minutes and a bus on line $B$ every 6 minutes. The time interval between an arrival of a bus for line $A$ and a bus for line $B$ is uniformly distributed between 0 and 4 minutes. Find the probability:

$N$ stars are randomly scattered, independently of each other, in a sphere of radius $R$.

A satellite moving on an orbit between two parallels $60^{\circ }$ north and $60^{\circ }$ south (latitude) is equally likely to land at any point between the two parallels. What is the probability $p$ that the satellite will land in the north above $30^{\circ }$?

In the equation $\lambda^{3}+3 X \lambda +Y = 0$, the coefficients $X, Y$ are uniformly distributed in the rectangle $\left|X\right| \leq a$, $\left|Y\right| \leq b$. What is the probability $p$ that the equation has real roots?

Let $P_{k,n}$ denote the probability of $k$ successes in $n$ independent Bernoulli trials. By using the generating function $G_{n}(t)$ of $P_{n,k}$ ($k = 0, 1, 2, \ldots$), deduce the binomial distribution.

$A$ and $B$ play the following game. They flip a coin, if the outcome is a head $A$ gets 1 dollar from $B$, otherwise $A$ pays 1 dollar to $B$. Initially, each of them has 3 dollars. The game terminates when either $A$ or $B$ loses all his money. What is the probability $p_{n}$ that $n$ flips are needed?

Let $Y_{n} = \max \left\{ X_{1}, \ldots , X_{n}\right\}$ where $X_{1}, \ldots , X_{n}$ are independent and identically distributed random variables with a uniform distribution on $(0,1)$. Show that the distribution of

converges (as $n \rightarrow \infty$) to the exponential distribution with distribution function $F(z) = 1-e^{-z}$.

A discrete random variable $X_{n}$ that appears in the theory of extremes has the distribution function

and $X_{n}$ takes the values $1/n, 2/n, \ldots$. Show that the sequence $\left\{ X_{n}\right\}$ converges in law to a continuous distribution with distribution function

If $X_{n} \xrightarrow {P} X$ and $\mathbb {E}\left[(X_{n}-Y_{n})^{2}\right] \rightarrow 0$, show that the sequence $\left\{ Y_{n}\right\}$ also converges to $X$ in probability.