Convergence of sequences of random variables and probability distributions

Let $\xi_{n} \xrightarrow {\mathbb {P}} \xi$ and $\xi_{n} \xrightarrow {\mathbb {P}} \eta$. Prove that $\mathbb {P}\left(\xi = \eta \right) = 1$.

Prove that if $\xi_{n}-a_{n} \xrightarrow {\mathbb {P}} 0$ and $\xi_{n}-b_{n} \xrightarrow {\mathbb {P}} 0$, where $a_{1}, a_{2}, \ldots$ and $b_{1}, b_{2}, \ldots$ are two sequences of real numbers, then $a_{n}-b_{n} \rightarrow 0$.

Let $\xi_{n} \xrightarrow {\mathbb {P}} \xi$ and $\eta_{n} \xrightarrow {\mathbb {P}} \eta$. Prove that:

Let $\xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables, where $\xi_{n}$ takes the values $e^{-a n}$ and $e^{a n}$ with probabilities $1-e^{-b n}$ and $e^{-b n}(b \geqslant 0)$ respectively. For which values of $a$ and $b$ does $\xi_{n} \xrightarrow {\mathbb {P}} 0$?

Let $\xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables such that $\mathbb {P}\left(\left|\xi_{n}\right| \geqslant c > 0\right) \geqslant \delta > 0, n = 1,2, \ldots$. Let $a_{1}, a_{2}, \ldots$ be a sequence of real numbers such that $a_{n} \xi_{n} \rightarrow 0$. Prove that $a_{n} \rightarrow 0$.

Let $\xi_{n}-a_{n} \xrightarrow {\mathbb {P}} 0$. Prove that $m \xi_{n}-a_{n} \rightarrow 0$ ($m \xi_{n}$ is the median of $\xi_{n}$).

Prove that a necessary and sufficient condition for a sequence of random variables $\xi_{1}, \xi_{2}, \ldots$ to converge in probability to some random variable $\xi$ is that for every $\varepsilon > 0$ there exists an $N$ such that for $n, m \geqslant N$

Let $\quad \xi_{n} \xrightarrow {\mathbb {P}} \xi , \eta_{n} \xrightarrow {\mathbb {P}} \eta$ and $\mathbb {P}\left(\xi = \eta \right) = 1$. Prove that for every $\varepsilon > 0$

as $n \rightarrow \infty$.

Let $\left(\xi_{n}-\xi \right)^{2} \xrightarrow {\mathbb {P}} 0$. Prove that $\xi_{n}^{2} \xrightarrow {\mathbb {P}} \xi^{2}$.

Let $\xi_{n} \xrightarrow {\mathbb {P}} a$ and let $f(x)$ be a Borel function having a derivative at the point $x = a$. Prove that

where $\eta_{n} \xrightarrow {\mathbb {P}} 0$.

Prove that if a sequence of random variables $\xi_{1}, \xi_{2}, \ldots$ converges almost surely, then the function $\xi (\omega )$, equal to $\lim_{n \rightarrow \infty } \xi_{n}$ if this limit exists, and to zero otherwise, is a random variable.

Let $\xi_{n} \rightarrow \xi$ a.s. and $\eta_{n} \rightarrow \eta$ a.s. Prove that:

Prove that $\xi_{n} \rightarrow \xi$ a.s. if and only if for every $\varepsilon > 0$

as $n \rightarrow \infty$.

Prove that $\xi_{n} \rightarrow \xi$ a.s. if and only if for every $\varepsilon > 0$

Prove that a necessary and sufficient condition for a sequence of random variables to converge with probability 1 to some random variable is that it be, with probability 1, a Cauchy sequence.

Prove that a necessary and sufficient condition for $\xi_{n} \rightarrow \xi$ a.s. is that for every $\varepsilon > 0$

as $n \rightarrow \infty$.

Prove that a necessary and sufficient condition for a sequence of random variables $\xi_{1}, \xi_{2}, \ldots$ to be, with probability 1, a Cauchy sequence, is that for every $\varepsilon > 0$

as $n \rightarrow \infty$.

Let $\varepsilon > 0, \xi , \xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables. Prove that if

then $\xi_{n} \rightarrow \xi$ a.s.

Let $\xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables. Prove that if for some summable sequence of positive numbers $\varepsilon_{1}, \varepsilon_{2}, \ldots$

then the sequence $\xi_{1}, \xi_{2}, \ldots$ converges with probability 1 to some almost surely finite random variable.

Let $\xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables. Suppose that there exist a random variable $\xi$ and a subsequence of positive integers $n_{1}, n_{2}, \ldots$ such that $\xi_{n_{k}} \rightarrow \xi$ a.s. and

as $k \rightarrow \infty$. Prove that then $\xi_{n} \rightarrow \xi$ a.s.

Prove that if $\xi_{m n} \rightarrow \xi_{m}$ a.s. as $n \rightarrow \infty$ and $\xi_{m} \rightarrow \xi$ a.s. as $m \rightarrow \infty$, then there exist two subsequences $\left\{ m_{k}\right\}$ and $\left\{ n_{k}\right\}$ such that $\xi_{m_{k} n_{k}} \rightarrow \xi$ a.s. as $k \rightarrow \infty$.

Prove that convergence with probability 1 implies convergence in probability.

Show that convergence in probability does not imply convergence with probability 1.

Let $\xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables. Prove that:

Prove that a necessary and sufficient condition for the notions of convergence with probability 1 and convergence in probability to coincide for some probability space is that this space be atomic.

Let $\mathscr {F}$ be the space of equivalence classes of random variables that coincide with probability 1. For any two classes $\Xi , H \in \mathscr {F}$ set

where $\xi$ and $\eta$ are random variables from $\Xi$ and $H$ respectively. Prove that $d$ is a metric on $\mathscr {F}$ and that convergence in probability is equivalent to convergence in the metric $d$.

Prove that a necessary and sufficient condition for a sequence of random variables to converge in mean of order $p \geqslant 1$ is that this sequence be a Cauchy sequence in mean of order $p$.

Prove that convergence in mean of some positive order implies convergence in probability.

Give an example showing that convergence in probability does not imply convergence in mean of order $p > 0$.

Give an example showing that convergence in mean of any positive order does not imply convergence with probability 1.

Give an example of a sequence of random variables that converges with probability 1 and is such that no subsequence of it converges in mean of order $p > 0$.

Prove that if $\xi_{n} \rightarrow \xi$ a.s. and for some $p > 0$, $\mathbb {E}\left[\left|\xi_{n}-\eta \right|^{p}\right] \rightarrow 0$, then $\mathbb {P}\left(\xi = \eta \right) = 1$.

Let $\mathbf{P}_{1}, \mathbf{P}_{2}, \ldots$ be a sequence of probability distributions, where $\mathbf{P}_{n}$ is concentrated at the point $x_{n}(n = 1,2, \ldots )$. Prove that weak convergence $\mathbf{P}_{n} \xrightarrow {W} \mathbb {P}$ means that the limit $\lim_{n \rightarrow \infty } x_{n} = x$ exists and the distribution $\mathbb {P}$ is concentrated at the point $x$. Prove the converse.

Let $\mathbb {P}, \mathbf{P}_{1}, \mathbf{P}_{2}, \ldots$ be a sequence of integer-valued distributions. Prove that $\mathbf{P}_{n} \xrightarrow {W} \mathbb {P}$ if and only if $\mathbb {P}_{n}\left(k\right) \rightarrow \mathbb {P}\left(k\right)$ for every integer $k$.

Let $\mathbb {P}$ be Lebesgue measure on the unit interval, and let $\mathbf{P}_{n}$ assign masses $\frac{1}{n}$ to certain points chosen one in each of the intervals $\left(\frac{i-1}{n}, \frac{i}{n}\right), i = 1, \ldots , n$. Prove that $\mathbf{P}_{n} \xrightarrow {W} \mathbb {P}$.

Let $F(x), F_{1}(x), F_{2}(x), \ldots$ be a sequence of distribution functions. Prove that if $F_{n}(x) \rightarrow F(x)$ for all $x$ in some dense set on the line, then $F_{n} \xrightarrow {W} F$.

Give an example of a sequence of probability distributions $\mathbb {P}, \mathbf{P}_{1}, \mathbf{P}_{2}, \ldots$ and a bounded function $f(x)$ such that $\mathbf{P}_{n} \xrightarrow {W} \mathbb {P}$, but the relation

does not hold.

Give an example of a sequence of probability distributions $\mathbb {P}, \mathbf{P}_{1}, \mathbf{P}_{2}, \ldots$ and a continuous function $f(x)$ such that $\mathbf{P}_{n} \xrightarrow {W} \mathbb {P}$, but the relation

does not hold.

Prove that $\mathbf{P}_{n} \xrightarrow {W} \mathbb {P}$ if and only if every subsequence $\left\{ \mathbf{P}_{n^{\prime }}\right\}$ of the sequence $\left\{ \mathbf{P}_{n}\right\}$ contains a subsequence $\left\{ \mathbf{P}_{n^{\prime \prime }}\right\}$ such that $\mathbf{P}_{n^{\prime \prime }} \xrightarrow {W} \mathbb {P}$.

Prove that if $F_{n} \xrightarrow {W} F$ and if $F(x)$ is continuous at every point of a closed set $A$, then

Prove that $F_{n} \xrightarrow {W} F$ if and only if

and

Let $\mathbb {P}, \mathbf{P}_{1}, \mathbf{P}_{2}, \ldots$ be a sequence of probability distributions. Prove that the following three conditions are equivalent:

1. $\mathbf{P}_{n} \xrightarrow {W} \mathbb {P}$,

2. for every closed set $B$

   $$\limsup _{n \rightarrow \infty } \mathbb {P}_{n}\left(B\right) \leqslant \mathbb {P}\left(B\right),$$

3. for every open set $G$

   $$\liminf _{n \rightarrow \infty } \mathbb {P}_{n}\left(G\right) \geqslant \mathbb {P}\left(G\right) .$$

Prove that a necessary and sufficient condition for $\mathbf{P}_{n} \xrightarrow {W} \mathbb {P}$ is that for every $\mathbb {P}$-continuous set $A$ the relation

holds ($\mathbb {P}$-continuous means a Borel set whose boundary satisfies $\mathbb {P}\left(\partial A\right) = 0$, i.e., has probability 0).

Let $\mathbb {P}, \mathbf{P}_{1}, \mathbf{P}_{2} \ldots$ be a sequence of probability distributions, and let for every bounded uniformly continuous function $f(x)$

Prove that $\mathbf{P}_{n} \xrightarrow {W} \mathbb {P}$.

Let $\mathbb {P}, \mathbf{P}_{1}, \mathbf{P}_{2}, \ldots$ be a sequence of probability distributions, and let for every bounded function $f(x)$ possessing continuous derivatives of every order,

Prove that $\mathbf{P}_{n} \xrightarrow {W} \mathbb {P}$.

Let $\xi , \xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables. Prove that $\xi_{n} \xrightarrow {D} \xi$ if and only if

for every continuous distribution function $F(x)$.

Let a random variable $\eta$ be independent of the random variables $\xi , \xi_{1}, \xi_{2}, \ldots$ and have distribution function $F(x)$. Prove that

if and only if

Let a sequence of distribution functions $F_{1}(x)$, $F_{2}(x)$,... converge weakly to some distribution function having at least two points of increase. Prove that the sequence $\left\{ F_{n}\left(a_{n} x+b_{n}\right)\right\}$ converges weakly to a distribution concentrated at zero if and only if $a_{n} \rightarrow \infty$ and $b_{n} = o\left(a_{n}\right)$, $n \rightarrow \infty$.

Let $F(x), F_{1}(x), F_{2}(x), \ldots$ be a sequence of distribution functions such that, for some $a_{n}, b_{n} > 0$,

Prove that if the sequences of real numbers $\alpha_{1}$, $\alpha_{2}, \ldots$ and $\beta_{1}, \beta_{2}, \ldots$ are such that

then the convergence

holds.

Let $\mathbb {P}, \mathbf{P}_{1}, \mathbf{P}_{2}, \ldots$ be a sequence of absolutely continuous probability distributions, with $p(x), p_{1}(x), p_{2}(x), \ldots$ the corresponding densities. Prove that if $p_{n}(x) \rightarrow p(x), n \rightarrow \infty$, uniformly on every finite interval, then $\mathbf{P}_{n} \xrightarrow {W} \mathbb {P}$. Is the converse true?

Let $\mathbf{R}_{n} \xrightarrow {W} \mathbf{R}, \mathbf{P}_{n} \xrightarrow {W} \mathbb {P}$ and $\mathbf{R}_{n} = \mathbf{P}_{n} * \mathbf{Q}_{n}$. Prove that there exists a distribution $\mathbf{Q}$ such that $\mathbf{R} = \mathbb {P} * \mathbf{Q}$.

Are the following families of probability distributions relatively compact:

Indicate which of the families of probability distributions given in the previous problem are tight.

(Prokhorov's theorem) Prove that a family of distributions on the line is relatively compact if and only if it is tight.

Prove that the set of all distributions whose expectation equals $a$ and whose variance equals $\sigma^{2}$ is relatively compact.

Prove that a family of normal distributions is tight if and only if the expectations and variances of the elements of this family are uniformly bounded.

Prove that a countable family of probability measures is tight if and only if the corresponding distribution functions satisfy the relations

uniformly in $n$.

Prove that weak convergence of distributions is equivalent to convergence in the Lévy metric (for the definition, see the introduction to Chapter 3)

Let the joint distribution of $\xi_{n}$ and $\eta_{n}$ converge weakly to the joint distribution of $\xi$ and $\eta$. Prove that the distribution of $\xi_{n}+\eta_{n}$ converges weakly to the distribution of $\xi +\eta$.

Prove that convergence in probability implies convergence in distribution.

Give an example showing that convergence in distribution does not imply convergence in probability.

Let $\xi_{n} \xrightarrow {D} a$, where $a$ is a constant. Prove that $\xi_{n} \xrightarrow {\mathbb {P}} a$.

Let $\xi_{n} \xrightarrow {D} \xi$. Is it true that $\xi_{n}-\xi \xrightarrow {D} 0$?

Prove that if $\xi_{n} \xrightarrow {D} \xi$ and $\eta_{n} \xrightarrow {\mathbb {P}} 0$, then:

Prove that if $\xi_{n} \xrightarrow {D} \xi ,\left|\xi_{n}-\eta_{n}\right| \leqslant \zeta_{n}\left|\xi_{n}\right|$ and $\zeta_{n} \xrightarrow {\mathbb {P}} 0$, then $\eta_{n} \xrightarrow {D} \xi$.

Prove that if $\xi_{n} \xrightarrow {D} \xi ,\left|\xi_{n}-\eta_{n}\right| \leqslant \zeta_{n}\left|\eta_{n}\right|$ and $\zeta_{n} \xrightarrow {\mathbb {P}} 0$, then $\eta_{n} \xrightarrow {D} \xi$.

Let $\xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables such that

What can be said about the sequence of distribution functions

Let the function $f(x)$ have a derivative at the point $x = 0$. Prove that if $\xi_{n} \eta_{n} \xrightarrow {D} \eta$ and $\eta_{n} \xrightarrow {\mathbb {P}} 0$, then

Let the functions $f(x), f_{1}(x), f_{2}(x), \ldots$ have continuous derivatives, with $f_{n}^{\prime }(x) \rightarrow f^{\prime }(x)$ uniformly in $x$. Prove that if $\xi_{n} \eta_{n} \xrightarrow {D} \eta$ and $\eta_{n} \xrightarrow {\mathbb {P}} 0$, then

Let $\xi , \xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables defined on a probability space $(\Omega , \mathscr {A}, \mathbb {P})$, with $\xi_{n}(\omega ) \rightarrow \xi (\omega )$ as $n \rightarrow \infty$ for every $\omega \in \Omega$. Give an example showing that $\mathbb {E}\left[\xi_{n}\right]$ need not converge to $\mathbb {E}\left[\xi \right]$ even when all these expectations exist.

Give an example showing that convergence in probability does not imply convergence of the expectations.

Let $\xi_{n} \xrightarrow {D} \xi$. Give an example where $\mathbb {E}\left[\xi_{n}\right]$ exist but $\mathbb {E}\left[\xi \right]$ does not, and conversely.

Let a sequence of random variables $\xi_{1}, \xi_{2}, \ldots$ converge in mean square to a random variable $\xi$, with $\mathbb {E}\left[\left|\xi_{n}\right|\right] < \infty , n = 1,2, \ldots$ Prove that $\mathbb {E}\left[\left|\xi \right|\right] < \infty$ and

Let a sequence of distribution functions $F_{1}(x)$, $F_{2}(x), \ldots$ converge weakly to a distribution function $F(x)$. Denote by $\sigma_{n}^{2}$ and $\sigma^{2}$ the variances of the distributions $F_{n}$ and $F$ respectively ($n = 1,2, \ldots$). Prove that

Let $F(x), F_{1}(x), F_{2}(x), \ldots$ be a sequence of distribution functions with expectations $a, a_{1}, a_{2}, \ldots$ and variances $\sigma^{2}, \sigma_{1}^{2}, \sigma_{2}^{2}, \ldots$ respectively. Prove that if $F_{n} \xrightarrow {W} F$ and $\sigma_{n}^{2} \rightarrow \sigma^{2}$, then $a_{n} \rightarrow a$.

Let a sequence of distribution functions $F_{1}(x)$, $F_{2}(x), \ldots$ with variances $\sigma_{1}^{2}, \sigma_{2}^{2}, \ldots$ converge weakly to a distribution function $F(x)$ with variance $\sigma^{2}$, and suppose the sequence $\sigma_{1}^{2}, \sigma_{2}^{2}, \ldots$ converges to some finite limit $b$. Must the equality $b = \sigma^{2}$ hold?

Let $\xi_{n} \xrightarrow {D} \xi$. Prove that

A sequence of random variables $\xi_{1}, \xi_{2}, \ldots$ is uniformly integrable. Prove that

Let $\xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables. Prove that if for some $\varepsilon > 0$

then the sequence $\xi_{1}, \xi_{2}, \ldots$ is uniformly integrable.

Let $\xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables and suppose there exists a random variable $\eta$ such that

and for every $a > 0$ and all $n$

Prove that the sequence $\xi_{1}, \xi_{2}, \ldots$ is uniformly integrable.

Let $\xi_{1}, \xi_{2}, \ldots$ be a uniformly integrable sequence of random variables and let $\xi_{n} \xrightarrow {D} \xi$. Prove that $\mathbb {E}\left[\xi_{n}\right] \rightarrow \mathbb {E}\left[\xi \right]$.

Let $\xi , \xi_{1}, \xi_{2}, \ldots$ be a sequence of nonnegative random variables with finite expectations. Prove that if $\xi_{n} \xrightarrow {D} \xi$ and $\mathbb {E}\left[\xi_{n}\right] \rightarrow \mathbb {E}\left[\xi \right]$, then the sequence $\xi_{1}$, $\xi_{2}, \ldots$ is uniformly integrable.

Prove that a sequence of random variables $\xi_{1}$, $\xi_{2}, \ldots$ is uniformly integrable if and only if

and for every $\varepsilon > 0$ there exists $\delta$ such that for all $n$, the inequality $\mathbb {P}\left(E\right) \leqslant \delta$ implies

Let $\xi , \xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables and

as $n \rightarrow \infty$. Must the convergence

hold?

8 A. V. Prokhorov et al.

Does the answer to the previous problem change if it is additionally known that $\xi_{n} \xrightarrow {\mathbb {P}} \xi$?

Let $\xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables, and let $f(x)$ be a positive measurable function satisfying the condition

Prove that if

then the sequence $\xi_{1}, \xi_{2}, \ldots$ is uniformly integrable.

Let $\xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables. Prove that if the sequence $\left|\xi_{1}\right|^{r},\left|\xi_{2}\right|^{r}, \ldots (r > 0)$ is uniformly integrable, then for every $0 < r^{\prime } < r$ the sequence $\left|\xi_{1}\right|^{r^{\prime }},\left|\xi_{2}\right|^{r^{\prime }}, \ldots$ is uniformly integrable.

Let $\xi_{n} \xrightarrow {D} \xi$ and for some $r > 0$

Prove that

for every positive $r^{\prime } < r$.

Let $\xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables, where $\xi_{n}$ takes the values $n^{\alpha }$ and 0 with probabilities $1 / n$ and $1-1 / n$ respectively ($n = 1,2, \ldots$). Investigate the convergence of the sequences $\left\{ \xi_{n}\right\}$ (in probability) and $\left\{ E\left|\xi_{n}\right|^{r}\right\}$ depending on the choice of $\alpha$ and $r$.

Let $\xi_{1}, \xi_{2}, \ldots$ be a sequence of nonnegative random variables such that $\xi_{n} \xrightarrow {\mathbb {P}} \xi$ and $\mathbb {E}\left[\xi_{n}\right] \rightarrow \mathbb {E}\left[\xi \right] < \infty$. Prove that $E\left|\xi_{n}-\xi \right| \rightarrow 0$.

Let $\xi_{1}, \xi_{2}, \ldots$ and $\eta_{1}, \eta_{2}, \ldots$ be two sequences of random variables such that $\mathbb {P}\left(\xi_{n} \geqslant \eta_{n} \geqslant 0\right) = 1, \quad \xi_{n} \xrightarrow {\mathbb {P}} \xi_{,} \eta_{n} \xrightarrow {\mathbb {P}} \eta$ and $\mathbb {E}\left[\xi_{n}\right] \rightarrow \mathbb {E}\left[\xi \right]$. Prove that $\mathbb {E}\left[\left|\eta_{n}-\eta \right|\right] \rightarrow 0$.

Let $\xi_{n} \rightarrow \xi$ in mean of order $r > 0$:

as $n \rightarrow \infty$. Prove that

Let $\xi_{1}, \xi_{2}, \ldots$ be a sequence of random variables such that for some $p > 0$

Prove that $\xi_{n} \rightarrow 0$ a.s.

Let $\xi_{n} \xrightarrow {\mathbb {P}} \xi$ and

Prove that the sequence $\xi_{1}, \xi_{2}, \ldots$ is uniformly integrable and

Let $\xi_{n} \xrightarrow {\mathbb {P}} \xi$ and

Prove that

Prove that if the sequence $\left|\xi_{1}\right|^{\delta },\left|\xi_{2}\right|^{\delta }, \ldots$ is uniformly integrable for some $\delta > 0$, then the sequence of distributions of the random variables $\xi_{1}, \xi_{2}, \ldots$ is tight.

Let $\xi_{1}, \xi_{2}, \ldots$ and $\eta_{1}, \eta_{2}, \ldots$ be two sequences of positive random variables. Can there exist positive numbers $\alpha$ and $\beta$ such that