Random Variables and Expected Values

$2.11 \uparrow$ A necessary and sufficient condition for a $\sigma$-field $\mathscr {G}$ to be countably generated is that $\mathscr {G}=\sigma (X)$ for some random variable $X$. Hint: If $\mathscr {G}= \sigma \left(A_{1}, A_{2}, \ldots \right)$, consider $X=\sum_{k=1}^{\infty } f\left(I_{A_{k}}\right) / 10^{k}$, where $f(x)$ is 4 for $x=0$ and 5 for $x \neq 0$.

If $X$ is a positive random variable with density $f$, then $X^{-1}$ has density $f(1 / x) / x^{2}$. Prove this by (20.16) and by a direct argument.

Suppose that a two-dimensional distribution function $F$ has a continuous density $f$. Show that $f(x, y)=\partial^{2} F(x, y) / \partial x \partial y$.

The construction in Theorem 20.4 requires only Lebesgue measure on the unit interval. Use the theorem to prove the existence of Lebesgue measure on $R^{k}$. First construct $\lambda_{k}$ restricted to $(-n, n] \times \cdot \times (-n, n]$, and then pass to the limit $(n \rightarrow \infty )$. The idea is to argue from first principles, and not to use previous constructions, such as those in Theorems 12.5 and 18.2.

Suppose that $A, B$, and $C$ are positive, independent random variables with distribution function $F$. Show that the quadratic $A z^{2}+B z+C$ has real zeros with probability $\int_{0}^{\infty } \int_{0}^{\infty } F\left(x^{2} / 4 y\right) d F(x) d F(y)$.

Show that $X_{1}, X_{2}, \ldots$ are independent if $\sigma \left(X_{1}, \ldots , X_{n-1}\right)$ and $\sigma \left(X_{n}\right)$ are independent for each $n$.

Let $X_{0}, X_{1}, \ldots$ be a persistent, irreducible Markov chain, and for a fixed state $j$ let $T_{1}, T_{2}, \ldots$ be the times of the successive passages through $j$. Let $Z_{1}=T_{1}$ and $Z_{n}=T_{n}-T_{n-1}, n \geq 2$. Show that $Z_{1}, Z_{2}, \ldots$ are independent and that $P\left[Z_{n}=k\right]=f_{j f}^{(k)}$ for $n \geq 2$.

Ranks and records. Let $X_{1}, X_{2}, \ldots$ be independent random variables with a common continuous distribution function. Let $B$ be the $\omega$-set where $X_{m}(\omega )=X_{n}(\omega )$ for some pair $m, n$ of distinct integers, and show that $\mathbb {P}\left(B\right)=0$. Remove $B$ from the space $\Omega$ on which the $X_{n}$ are defined. This leaves the joint distributions of the $X_{n}$ unchanged and makes ties impossible.

Let $T^{(n)}(\omega )=\left(T_{1}^{(n)}(\omega ), \ldots , T_{n}^{(n)}(\omega )\right)$ be that permutation $\left(t_{1}, \ldots , t_{n}\right)$ of $(1, \ldots , n)$ for which $X_{t_{1}}(\omega )<X_{t_{2}}(\omega )<\cdots <X_{t_{n}}(\omega )$. Let $Y_{n}$ be the rank of $X_{n}$ among $X_{1}, \ldots , X_{n}: Y_{n}=r$ if and only if $X_{i}<X_{n}$ for exactly $r-1$ values of $i$ preceding $n$.

$\uparrow$ Record values. Let $A_{n}$ be the event that a record occurs at time $n$ : $\max_{k<n} X_{k}<X_{n}$.

Use Fubini's theorem to prove that convolution of finite measures is commutative and associative.

Suppose that $X$ and $Y$ are independent and have densities. Use (20.20) to find the joint density for $(X+Y, X)$ and then use (20.19) to find the density for $X+Y$. Check with (20.38).

If $F(x-\epsilon )<F(x+\epsilon )$ for all positive $\epsilon$, then $x$ is a point of increase of $F$ (see Problem 12.9). If $F(x-)<F(x)$, then $x$ is an atom of $F$.

Suppose that $\mu$ and $\nu$ consist of masses $\alpha_{n}$ and $\beta_{n}$ at $n, n=0,1,2, \ldots$. Show that $\mu * \nu$ consists of a mass of $\sum_{k=0}^{n} \alpha_{k} \beta_{n-k}$ at $n, n=0,1,2, \ldots$. Show that two Poisson distributions (the parameters may differ) convolve to a Poisson distribution.

The Cauchy distribution has density

for $u>0$. (By (17.9), the density integrates to 1.)

$18.18 \uparrow$ Let $X_{1}, \ldots , X_{n}$ be independent, each having the standard normal distribution Show that

has density

over $(0, \infty )$. This is called the chi-squared distribution with $n$ degrees of freedom.

↑ The gamma distribution has density

over $(0, \infty )$ for positive parameters $\alpha$ and $u$. Check that (20.47) integrates to 1 Show that

Note that (20.46) is $f\left(x ; \frac{1}{2}, n / 2\right)$, and from (20.48) deduce again that (20.46) is the density of $\chi_{n}^{2}$. Note that the exponential density (20.10) is $f(x ; \alpha , 1)$, and from (20.48) deduce (20.39) once again.

$\uparrow$ Let $N, X_{1}, X_{2}, \ldots$ be independent, where $\mathbb {P}\left(N=n\right)=q^{n-1} p, n \geq 1$, and each $X_{k}$ has the exponential density $f(x ; \alpha , 1)$. Show tnat $X_{1}+\cdots +X_{N}$ has density $f(x ; \alpha p, 1)$.

Let $A_{n m}(\epsilon )=\left[\left|Z_{k}-Z\right|<\epsilon , n \leq k \leq m\right]$. Show that $Z_{n} \rightarrow Z$ with probability 1 if and only if $\lim_{n} \lim_{m} \mathbb {P}\left(A_{n m}(\epsilon )\right)=1$ for all positive $\epsilon$, whereas $Z_{n} \rightarrow_{p} Z$ if and only if $\lim_{n} \mathbb {P}\left(A_{n \eta }(\epsilon )\right)=1$ for all positive $\epsilon$.

Suppose that the sequence $\left(X_{n}\right)$ is fundamental in probability in the sense that for $\epsilon$ positive there exists an $N_{\epsilon }$ such that $\mathbb {P}\left(\left|X_{m}-X_{n}\right|>\epsilon \right)<\epsilon$ for $m, n>N_{\epsilon }$.

If $X_{n} \rightarrow 0$ with probability 1, then $n^{-1} \sum_{k=1}^{n} X_{k} \rightarrow 0$ with probability 1 by the standard theorem on Cesàro means [A30]. Show by example that this is not so if convergence with probability 1 is replaced by convergence in probability.

$20.21, 20.24 \uparrow$ Let $d(X, Y)$ be the infimum of those positive $\epsilon$ for which $\mathbb {P}\left(\left|X-Y\right| \geq \epsilon \right) \leq \epsilon$.

Construct in $R^{k}$ a random variable $X$ that is uniformly distributed over the surface of the unit sphere in the sense that $\left|X\right|=1$ and $U X$ has the same distribution as $X$ for orthogonal transformations $U$. Hint Let $Z$ be uniformly distributed in the unit ball in $R^{k}$, define $\psi (x)=x /\left|x\right|(\psi (0)=(1,0, \quad 0)$, say), and take $X=\psi (Z)$.

$\uparrow$ Let $\Theta$ and $\Phi$ be the longitude and latitude of a random point on the surface of the unit sphere in $R^{3}$. Show that $\Theta$ and $\Phi$ are independent, $\Theta$ is uniformly distributed over $[0,2 \pi )$, and $\Phi$ is distributed over $[-\pi / 2,+\pi / 2]$ with density $\frac{1}{2} \cos \phi$

Prove

differentiate $k$ times with respect to $t$ inside the integral (justify), and derive (21.7) again.

Show that, if $X$ has the standard normal distribution, then $\mathbb {E}\left[|X|^{2 n+1}\right]= 2^{n} n!\sqrt{2 / \pi }$.

$20.9 \uparrow$ Records. Consider the sequence of records in the sense of Problem 20.9. Show that the expected waiting time to the next record is infinite.

20.14 ↑ Show that the Cauchy distribution has no mean.

Prove the first Borel-Cantelli lemma by applying Theorem 16.6 to indicator random variables. Why is Theorem 16.6 not enough for the second Borel-Cantelli lemma?

Prove ( 21.9 ) by Fubini's theorem.

Prove for integrable $X$ that

Suppose that $X$ and $Y$ are random variables with distribution functions $F$ and $G$.

↑ Let $X, Y$, and $Z$ be independent random variables such that $X$ and $Y$ assume the values $0,1,2$ with probability $\frac{1}{3}$ each and $Z$ assumes the values 0 and 1 with probabilities $\frac{1}{3}$ and $\frac{2}{3}$. Let $X^{\prime }=X$ and $Y^{\prime }=X+Z(\bmod 3)$.

Suppose that $X$ and $Y$ are independent, nonnegative random variables and that $\mathbb {E}\left[X\right]=\infty$ and $\mathbb {E}\left[Y\right]=0$. What is the value common to $\mathbb {E}\left[X Y\right]$ and $\mathbb {E}\left[X\right] \mathbb {E}\left[Y\right]$? Use the conventions (15.2) for both the product of the random variables and the product of their expected values. What if $\mathbb {E}\left[X\right]=\infty$ and $0<\mathbb {E}\left[Y\right]<\infty$?

Suppose that $X$ and $Y$ are independent and that $f(x, y)$ is nonnegative. Put $g(x)=\mathbb {E}\left[f(x, Y)\right]$ and show that $\mathbb {E}\left[g(X)\right]=\mathbb {E}\left[f(X, Y)\right]$. Show more generally that $\int_{X \in A} g(X) d P=\int_{X \in A} f(X, Y) d P$. Extend to $f$ that may be negative.

$\uparrow$ The integrability of $X+Y$ does not imply that of $X$ and $Y$ separately. Show that it does if $X$ and $Y$ are independent.

$20.25 \uparrow$ Write $d_{1}(X, Y)=\mathbb {E}\left[\left|X-Y\right| /(1+\left|X-Y\right|)\right]$. Show that this is a metric equivalent to the one in Problem 20.25.

For the density $C \exp \left(-|x|^{1 / 2}\right),-\infty <x<\infty$, show that moments of all orders exist but that the moment generating function exists only at $s=0$.

16.6 ↑ Show that a moment generating function $M(s)$ defined in $\left(-s_{0}, s_{0}\right)$, $s_{0}>0$, can be extended to a function analytic in the strip $\left[z:-s_{0}<\operatorname {Re} z<s_{0}\right]$. If $M(s)$ is defined in $\left[0, s_{0}\right), s_{0}>0$, show that it can be extended to a function continuous in $\left[z: 0 \leq \operatorname {Re} z<s_{0}\right]$ and analytic in $\left[z \cdot 0<\operatorname {Re} z<s_{0}\right]$.

Use (21.28) to find the generating function of (20.39).

For independent random variables having moment generating functions, show by (21.28) that the variances add.

20.17 Show that the gamma density (20.47) has moment generating function $(1-s / \alpha )^{-u}$ for $s<\alpha$. Show that the $k$ th moment is $u(u+1) \cdots (u+k-$ 1) $/ \alpha^{k}$ Show that the chi-squared distribution with $n$ degrees of freedom has mean $n$ and variance $2 n$.

Let $X_{1}, X_{2}, \ldots$ be identically distributed random variables with finite second moment. Show that $n \mathbb {P}\left(\left|X_{1}\right| \geq \epsilon \sqrt{n}\right) \rightarrow 0$ and $n^{-1 / 2} \max_{k \leq n}\left|X_{k}\right| \rightarrow_{p} 0$.

Suppose that $X_{1}, X_{2}, \ldots$ is an independent sequence and $Y$ is measurable $\sigma \left(X_{n}, X_{n+1}, \ldots \right)$ for each $n$. Show that there exists a constant $a$ such that $\mathbb {P}\left(Y=a\right)=1$.

Assume $\left\{ X_{n}\right\}$ independent, and define $X_{n}^{(c)}$ as in Theorem 22.8. Prove that for $\sum \left|X_{n}\right|$ to converge with probability 1 it is necessary that $\sum \mathbb {P}\left(\left|X_{n}\right|>c\right)$ and $\sum \mathbb {E}\left[\mid X_{n}^{(c)}\right] \mid$ converge for all positive $c$ and sufficient that they converge for some positive $c$. If the three series (22.13) converge but $\Sigma \mathbb {E}\left[\left|X_{n}^{(c)}\right|\right]=\infty$, then there is probability 1 that $\sum X_{n}$ converges conditionally but not absolutely.

Show under the hypothesis of Theorem 22.6 that $\sum X_{n}$ has finite variance and extend Theorem 22.4 to infinite sequences.

$20.1422 .1 \uparrow$ Suppose that $X_{1}, X_{2}, \ldots$ are independent, each with the Cauchy distribution (20.45) for a common value of $u$.

If $X_{1}, X_{2}, \ldots$ are independent and identically distributed, and if $\mathbb {P}\left(X_{1} \geq 0\right)=1$ and $\mathbb {P}\left(X_{1}>0\right)>0$, then $\sum_{n} X_{n}=\infty$ with probability 1. Deduce this from Theorem 22.1 and its corollary and also directly: find a positive $\epsilon$ such that $X_{r}>c$ infinitely often with probability 1.

Suppose that $X_{1}, X_{2}, \ldots$ are independent and identically distributed and $\mathbb {E}\left[\left|X_{1}\right|\right]=\infty$. Use (21.9) to show that $\sum_{n} \mathbb {P}\left(\left|X_{n}\right| \geq a n\right)=\infty$ for each $a$, and conclude that $\sup_{n} n^{-1}\left|X_{n}\right|=\infty$ with probability 1. Now show that $\sup_{n} n^{-1}\left|S_{n}\right| =\infty$ with probability 1. Compare with the corollary to Theorem 22.1.

Wald's equation. Let $X_{1}, X_{2}, \ldots$ be independent and identically distributed with finite mean, and put $S_{n}=X_{1}+\cdots +X_{n}$. Suppose that $\tau$ is a stopping time: $\tau$ has positive integers as values and $[\tau =n] \in \sigma \left(X_{1}, \ldots , X_{n}\right)$; see Section 7 for examples. Suppose also that $\mathbb {E}\left[\tau \right]<\infty$.

$20.9 \uparrow$ Let $Z_{n}$ be 1 or 0 according as at time $n$ there is or is not a record in the sense of Problem 20.9. Let $R_{n}=Z_{1}+\cdots +Z_{n}$ be the number of records up to time $n$. Show that $R_{n} / \log n \rightarrow_{P} 1$.

Suppose that $X_{0}, X_{1}, \ldots$ are independent and each is uniformly distributed over $[0,2 \pi ]$. Show that with probability 1 the series $\sum_{n} e^{i X_{n}} z^{n}$ has the unit circle as its natural boundary.

Prove (what is essentially Kolmogorov's zero-one law) that if $A$ is independent of a $\pi$-system $\mathscr {P}$ and $A \in \sigma (\mathscr {P})$, then $\mathbb {P}\left(A\right)$ is either 0 or 1.

Suppose that $\mathscr {A}$ is a semiring containing $\Omega$.

$22.12 \uparrow$ Burstin's theorem. Let $f$ be a Borel function on $[0,1]$ with arbitrarily small periods: For each $\epsilon$ there is a $p$ such that $0<p<\epsilon$ and $f(x)=f(x+p)$ for $0 \leq x \leq 1-p$. Show that such an $f$ is constant almost everywhere:

Assume that $X_{1}, \ldots , X_{n}$ are independent and $s, t, \alpha$ are nonnegative Let

Show that the minimum of independent exponential waiting times is again exponential and that the parameters add.

20.17 ↑ Show that the time $S_{n}$ of the $n$th event in a Poisson stream has the gamma density $f(x ; \alpha , n)$ as defined by (20.47). This is sometimes called the Erlang density.

Let $A_{i}=t-S_{N_{1}}$ be the time back to the most recent event in the Poisson stream (or to 0 ), and let $B_{i}=S_{N_{t}+1}-t$ be the time forward to the next event. Show that $A_{t}$ and $B_{t}$ are independent, that $B_{t}$ is distributed as $X_{1}$ (exponentially with parameter $\alpha$ ), and that $A_{t}$ is distributed as $\min \left\{ X_{1}, t\right\} : \mathbb {P}\left(A_{t} \leq i\right)$ is $0,1-e^{-\alpha x}$, or 1 as $x<0,0 \leq x<t$, or $x \geq t$.

$\uparrow$ Let $L_{t}=A_{t}+B_{t}=S_{N_{t}+1}-S_{N_{t}}$ be the length of the interarrival interval covering $t$.