Weak Convergence of Measures. The Invariance Principle

Prove that the measures $\delta_{x_{n}}$ defined on the Borel $\sigma$-algebra of a metric space $(\mathrm{S}, \rho )$ have a weak limit if and only if there exists an element $x \in \mathrm{S}$ such that $x_{n} \stackrel{\rho }{\longrightarrow } x$ as $n \rightarrow \infty$ (then $\delta_{x_{n}} \Rightarrow \delta_{x}$ as $n \rightarrow \infty )$.

Prove that $\mathbb {Q}_{n} \Rightarrow \mathbb {Q}$ as $n \rightarrow \infty$ (the measures are defined on $\mathscr {B}(\mathrm{S})$, where $(\mathrm{S}, \rho )$ is a metric space) if and only if property (1) holds for all real bounded Lipschitz functions, i.e., for functions $f: S \rightarrow \mathbb {R}$ such that

Definition 1. Let $(\mathrm{S}, \rho )$ be a metric space with Borel $\sigma$-algebra $\mathscr {B}(\mathrm{S})$, and let $\mathbb {Q}, \mathbb {Q}_{n}$ ($n \geqslant 1$) be measures on $(\mathrm{S}, \mathscr {B}(\mathrm{S}))$. The measures $\mathbb {Q}_{n}$ are said to converge weakly to the measure $\mathbb {Q}$ (notation: $\mathbb {Q}_{n} \Rightarrow \mathbb {Q}$) if for every function $f \in C_{b}(\mathrm{S}, \mathbb {R})$, i.e., every continuous bounded function $f: \mathrm{S} \rightarrow \mathbb {R}$,

Let $f: \mathrm{S} \rightarrow \mathrm{V}$ be a function, where $\mathrm{S}, \mathrm{V}$ are metric spaces (measurability of $f$ is not assumed). Show that the set $D_{f} = \left\{ x : \text{the function } f \text{ is discontinuous at the point } x\right\} \in \mathscr {B}(\mathrm{S})$. Prove that $\mathbb {Q}_{n} \Rightarrow \mathbb {Q}$ if and only if property (1) holds for every bounded function $f \in \mathscr {B}(\mathrm{S}) \mid \mathscr {B}(\mathbb {R})$ such that $\mathbb {Q}\left(D_{f}\right) = 0$.

Definition 1. Let $(\mathrm{S}, \rho )$ be a metric space with Borel $\sigma$-algebra $\mathscr {B}(\mathrm{S})$, and let $\mathbb {Q}, \mathbb {Q}_{n}$ ($n \geqslant 1$) be measures on $(\mathrm{S}, \mathscr {B}(\mathrm{S}))$. The measures $\mathbb {Q}_{n}$ are said to converge weakly to the measure $\mathbb {Q}$ (notation: $\mathbb {Q}_{n} \Rightarrow \mathbb {Q}$) if for every function $f \in C_{b}(\mathrm{S}, \mathbb {R})$, i.e., every continuous bounded function $f: \mathrm{S} \rightarrow \mathbb {R}$,

Let $S$ be the space $\mathbb {R}^{m}$ with the Euclidean metric. Is it true that $\mathbb {Q}_{n} \Rightarrow \mathbb {Q}$ if property (1) holds for the functions $f \in C_{0}^{\infty }\left(\mathbb {R}^{m}\right)$, i.e., for all infinitely differentiable functions with compact support?

Definition 1. Let $(\mathrm{S}, \rho )$ be a metric space with Borel $\sigma$-algebra $\mathscr {B}(\mathrm{S})$, and let $\mathbb {Q}, \mathbb {Q}_{n}$ ($n \geqslant 1$) be measures on $(\mathrm{S}, \mathscr {B}(\mathrm{S}))$. The measures $\mathbb {Q}_{n}$ are said to converge weakly to the measure $\mathbb {Q}$ (notation: $\mathbb {Q}_{n} \Rightarrow \mathbb {Q}$) if for every function $f \in C_{b}(\mathrm{S}, \mathbb {R})$, i.e., every continuous bounded function $f: \mathrm{S} \rightarrow \mathbb {R}$,

The definition of weak convergence (1) applies to arbitrary finite (not necessarily probability) measures as well.

Definition 1. Let $(\mathrm{S}, \rho )$ be a metric space with Borel $\sigma$-algebra $\mathscr {B}(\mathrm{S})$, and let $\mathbb {Q}, \mathbb {Q}_{n}$ ($n \geqslant 1$) be measures on $(\mathrm{S}, \mathscr {B}(\mathrm{S}))$. The measures $\mathbb {Q}_{n}$ are said to converge weakly to the measure $\mathbb {Q}$ (notation: $\mathbb {Q}_{n} \Rightarrow \mathbb {Q}$) if for every function $f \in C_{b}(\mathrm{S}, \mathbb {R})$, i.e., every continuous bounded function $f: \mathrm{S} \rightarrow \mathbb {R}$,

Let $\mathbb {Q}, \mathbb {Q}_{n}$ ($n \geqslant 1$) be finite measures on $(\mathrm{S}, \mathscr {B}(\mathrm{S}))$. Prove that Theorem 1 remains valid if the requirement $\mathbb {Q}_{n}\left(\mathrm{S}\right) \rightarrow \mathbb {Q}\left(\mathrm{S}\right)$ as $n \rightarrow \infty$ is added to its hypotheses.

Theorem 1 (A. D. Aleksandrov). Let $\mathbb {Q}, \mathbb {Q}_{n}$ ($n \geqslant 1$) be measures on a metric space $(\mathrm{S}, \rho )$. Then the weak convergence $\mathbb {Q}_{n} \Rightarrow \mathbb {Q}$ ($n \rightarrow \infty$) is equivalent to each of the following statements:

1. $\limsup_{n \rightarrow \infty } \mathbb {Q}_{n}\left(F\right) \leqslant \mathbb {Q}\left(F\right)$ for every closed set $F \in \mathscr {B}(\mathrm{S})$;

2. $\liminf_{n \rightarrow \infty } \mathbb {Q}_{n}\left(G\right) \geqslant \mathbb {Q}\left(G\right)$ for every open set $G \in \mathscr {B}(\mathrm{S})$;

3. $\lim_{n \rightarrow \infty } \mathbb {Q}_{n}\left(B\right) = \mathbb {Q}\left(B\right)$ for every set $B \in \mathscr {B}(\mathrm{S})$ such that $\mathbb {Q}\left(\partial B\right) = 0$.

Definition 8. In a measurable space $(\mathrm{S}, \mathscr {A})$, a class of sets $\mathscr {G} \subset \mathscr {A}$ is called a measure-determining class if the equality of any two measures on $\mathscr {G}$ implies their equality on $\mathscr {A}$. Let $\mathrm{S}$ be a metric space. A class of sets $\mathscr {M} \subset \mathscr {B}(\mathrm{S})$ is called a class determining (weak) convergence if for every sequence of measures $\mathbb {Q}, \mathbb {Q}_{1}, \mathbb {Q}_{2}, \ldots$ such that $\mathbb {Q}_{n}\left(B\right) \rightarrow \mathbb {Q}\left(B\right)$ for all $B \in \mathscr {M}$ with $\mathbb {Q}\left(\partial B\right) = 0$, we have $\mathbb {Q}_{n} \Rightarrow \mathbb {Q}$.

Show that a class of sets determining (weak) convergence is a measure-determining class. Explain why the converse statement need not hold.

(compare with Exercise 6 and Example 1) Prove that in the Polish space $\mathbb {R}^{\infty }$ endowed with a metric of the form

the Borel $\sigma$-algebra coincides with the cylindrical one, and the convergence $\mathbb {Q}_{n} \Rightarrow \mathbb {Q}$ as $n \rightarrow \infty$ holds if and only if all finite-dimensional distributions converge weakly. In other words, the class of cylinders in $\mathbb {R}^{\infty }$ is simultaneously both a measure-determining class and a convergence-determining class.

(the Cramér-Wold method). Prove that $\xi_{n} \stackrel{\mathscr {D}}{\rightarrow } \xi$ in $\left(\mathbb {R}^{m}, \mathscr {B}\left(\mathbb {R}^{m}\right)\right)$ if and only if $\left(a, \xi_{n}\right) \stackrel{\mathscr {D}}{\rightarrow }(a, \xi )$ as $n \rightarrow \infty$ for every nonrandom vector $a \in \mathbb {R}^{m}$, where $(\cdot , \cdot )$ is the inner product in $\mathbb {R}^{m}$.

Let $\mathrm{S}$ be a real separable Banach space, $\mathrm{S}^{*}$ the dual space (i.e., the space of continuous linear functionals on $\mathrm{S}$) and $\mathscr {P}(\mathrm{S})$ the space of probability measures on $\mathscr {B}(\mathrm{S})$.

Definition 9. The characteristic functional of a measure $\mathbb {Q} \in \mathscr {P}(\mathrm{S})$ (or of a random element $X: \Omega \rightarrow \mathrm{S}, X \in \mathscr {F} \mid \mathscr {B}(\mathrm{S})$, such that $\operatorname {Law}(X) = \mathbb {Q}$ ) is the mapping $\varphi_{\mathbb {Q}}: \mathrm{S}^{*} \rightarrow \mathbb {C}$ given by the formula

where $\left\langle y, x^{*} \right\rangle$ denotes the action of the linear functional $x^{*}$ on the element $y \in \mathrm{S}$.

Let $\mathscr {C}$ be the collection of cylinder sets in $\mathrm{S}$, i.e., sets of the form

where $B \in \mathscr {B}\left(\mathbb {R}^{n}\right), z_{1}^{*}, \ldots , z_{n}^{*} \in \mathrm{S}^{*}$.

(compare with Theorem 13). Show that $\mathscr {C}$ generates $\mathscr {B}(\mathrm{S})$. Prove that if $\mathbb {Q}_{n} \Rightarrow \mathbb {Q}$ in $\mathscr {P}(\mathrm{S})$, then $\varphi_{\mathbb {Q}_{n}} \rightarrow \varphi_{\mathbb {Q}}$ pointwise. Conversely, let $\varphi_{\mathbb {Q}_{n}} \rightarrow \varphi$ pointwise, where $\varphi : \mathrm{S}^{*} \rightarrow \mathbb {C}$ and the family $\left\{ \mathbb {Q}_{n}, n \geqslant 1\right\}$ is tight. Then $\varphi = \varphi_{\mathbb {Q}}$ for some $\mathbb {Q} \in \mathscr {P}(\mathrm{S})$ and $\mathbb {Q}_{n} \Rightarrow \mathbb {Q}$.

For a random vector $\xi$ in $\mathbb {R}^{m}$, let $F_{\xi }$ and $\varphi_{\xi }$ denote its distribution function and characteristic function, respectively (see § 14 of Chapter I).

Theorem 13. The relation $\xi_{n} \stackrel{\mathscr {D}}{\rightarrow } \xi$ for random vectors in $\mathbb {R}^{m}$ is equivalent to each of the following two conditions:

1. $F_{\xi_{n}}(x) \rightarrow F_{\xi }(x)$ as $n \rightarrow \infty$ at every point $x \in \mathbb {R}^{m}$ at which the function $F_{\xi }$ is continuous;

2. $\varphi_{\xi_{n}}(\lambda ) \rightarrow \varphi_{\xi }(\lambda )$ as $n \rightarrow \infty$ for every $\lambda \in \mathbb {R}^{m}$. Moreover, if for random vectors $\xi_{n}$ in $\mathbb {R}^{m}$ the convergence $\varphi_{\xi_{n}}(\lambda ) \rightarrow \varphi (\lambda )$ holds for all $\lambda \in \mathbb {R}^{m}$, where the function $\varphi$ is continuous at the point $0 \in \mathbb {R}^{m}$, then $\varphi (\cdot ) = \varphi_{\xi }(\cdot )$ for some random vector $\xi$ and $\xi_{n} \xrightarrow {\mathscr {D}} \xi$ as $n \rightarrow \infty$.

Using Theorem 13 of Chapter III, prove the invariance principle (Theorem 8) for the triangular array generated by a sequence of independent identically distributed random variables with zero mean and unit variance.

Theorem 13 of Chapter III (Skorokhod). Let $X_{1}, X_{2}, \ldots$ be a sequence of centered independent random variables defined on some probability space. Then there exists a probability space on which one can find a sequence of random variables $\left\{ T_{k}\right\}_{k \geqslant 1}$ and a Brownian motion $W = \left\{ W(t), t \geqslant 0\right\}$ such that

where the nonnegative variables $T_{k}-T_{k-1}, k \in \mathbb {N}\left(T_{0} \equiv 0\right)$, are independent. If $\mathbb {E}\left[X_{k}^{2}\right] < \infty$, then in addition to the indicated properties of the sequence $\left\{ T_{k}\right\}_{k \geqslant 1}$ one can ensure the relation $\mathbb {E}\left[T_{k}-T_{k-1}\right] = \mathbb {E}\left[X_{k}^{2}\right], k \in \mathbb {N}$.

Let $(\Omega , \mathscr {F}, \mathbb {P})$ be a probability space and let $X_{n, i}, i = 1, \ldots , m_{n}, n \geqslant 1$, be series of (for each $n$) independent real random variables defined on this space such that $\mathbb {E}\left[X_{n, i}\right] = 0$, $i = 1, \ldots , m_{n}$, and $\sum_{i = 1}^{m_{n}} \sigma_{n, i}^{2} = 1$, where $\sigma_{n, i}^{2} = \mathbb {E}\left[X_{n, i}^{2}\right]$. Denote $t_{n, 0} = 0, t_{n, j} = \sum_{i = 1}^{j} \sigma_{n, i}^{2}$ (then $t_{n, m_{n}} = 1$), $S_{n, 0} = 0, S_{n, j} = \sum_{i = 1}^{j} X_{n, i}$, $j = 1, \ldots , m_{n}$. Now, for each $\omega \in \Omega$, define the function $S_{n}(t, \omega ), t \in [0,1]$, as the continuous polygonal line with vertices $\left(t_{n, j}, S_{n, j}\right), j = 0, \ldots , m_{n}$, i.e., let

Here and below we assume that all $\sigma_{n, i} > 0$; in other words, we exclude from consideration the variables $X_{n, i} = 0$ a.s. Let $\mathbb {P}_{n}$ be the probability distribution of $S_{n}$ on $\mathscr {B}(C[0,1])$.

Theorem 8 (Prokhorov). Suppose that for the variables $X_{n, i}$ described above, where $n \geqslant 1$ and $i = 1, \ldots , m_{n}$, the Lindeberg condition holds:

Then $\mathbb {P}_{n} \Rightarrow \mathbb {W}$ ($n \rightarrow \infty$), where $\mathbb {W}$ is the Wiener measure. In other words, $S_{n} \stackrel{\mathscr {D}}{\rightarrow } W$ as $n \rightarrow \infty$, where $W = \left\{ W(t), t \in [0,1]\right\}$ is the Wiener process on the interval $[0,1]$.

Let $X_{1}, X_{2}, \ldots$ be independent identically distributed random variables, $\mathbb {P}\left(X_{1} = -1\right) = \mathbb {P}\left(X_{1} = 1\right) = 1 / 2$.

(compare with Corollary 1 of Chapter III). Prove that for every nonnegative integer $j$

where $S_{0} = 0, S_{k} = X_{1}+\cdots +X_{k}, k \geqslant 1$, and $X_{1}, X_{2}, \ldots$ are the random variables introduced above.

Corollary 1 of Chapter III (Bachelier). For all $t, y \geqslant 0$

i.e., $\operatorname {Law}(M(t)) = \operatorname {Law}(\left|W(t)\right|)$ for every $t \geqslant 0$. Here $W = \left\{ W(t), t \geqslant 0\right\}$ is a Brownian motion and $M(t, \omega ) = \sup_{s \in [0, t]} W(s, \omega )$, $t \geqslant 0$.

Let $\xi_{n} \xrightarrow {\mathscr {D}} \xi$, where $\xi , \xi_{n}, n \geqslant 1$, are real random variables, and the function $F_{\xi }(x)$ is continuous on $\mathbb {R}$. Then

Let $X_{1}, X_{2}, \ldots$ be independent identically distributed random variables, $\mathbb {P}\left(X_{1} = -1\right) = \mathbb {P}\left(X_{1} = 1\right) = 1 / 2$.

Prove that for the two-valued random variables $X_{i}$, $i \in \mathbb {N}$, introduced before Exercise 11, and their sums $S_{k} = X_{1}+\cdots +X_{k}, k \geqslant 1$, we have

Let us consider the space $C(T, \mathrm{S})$, where $\mathrm{S}$ is a metric space and $T$ is a locally compact Hausdorff space satisfying the second axiom of countability (the latter means that the space has a countable base). We endow $C(T, \mathrm{S})$ with the topology of uniform convergence on compact sets by introducing the metric

where $x = (x(t), t \in T), y = (y(t), t \in T) \in C(T, \mathrm{S})$, and $T$ can be represented as

Prove that $X^{(n)} \stackrel{\mathscr {D}}{\rightarrow } X$ in the space $C(T, \mathrm{S})$ introduced above if and only if $X_{\mid K}^{(n)} \stackrel{\mathscr {D}}{\rightarrow } X_{\mid K}$ in $C(K, \mathrm{S})$ for every compact set $K \subset T$; here, as before, $Y_{\mid K}$ denotes the passage from the function $Y = \left\{ Y_{t}, t \in T\right\}$ to its restriction $Y_{\mid K} = \left\{ Y_{t}, t \in K\right\}$.

Let $\xi_{1}, \xi_{2}, \ldots$ be independent identically distributed random vectors in $\mathbb {R}^{m}$ such that $\mathbb {E}\left[\xi_{1}\right] = 0, \mathbb {E}\left[\left\| \xi_{1}\right\|^{2}\right] < \infty$, where $\left\| \cdot \right\|$ is the Euclidean norm. Then the processes

where $\left[\cdot \right]$ is the integer part of a number, converge in distribution as $n \rightarrow \infty$ in the space $C\left([0, \infty ), \mathbb {R}^{m}\right)$ to a standard $m$-dimensional Brownian motion multiplied by a constant.

Let $X^{(n)}, n \geqslant 1,$ be continuous processes on $\mathbb {R}^{d}$ taking values, for each $t \in \mathbb {R}^{d}$, in a metric space $\mathrm{S}$. Prove that the distributions of these processes are tight in $C\left(\mathbb {R}^{d}, \mathrm{S}\right)$ if, for some constants $\alpha , \beta > 0$,

where $\left\| \cdot \right\|$ is some norm on $\mathbb {R}^{d}$.

The Skorokhod spaces $D\left([0,1]^{q}\right)$ and $D\left([0, \infty )^{q}\right), q \geqslant 1$, consist of real-valued functions defined on $[0,1]^{q}$ and $[0, \infty )^{q}$, respectively, that are continuous from above at every point and have a limit from below at every point $t \neq 0$ (the limit of a function $f$ from above at a $q$-dimensional point $t = \left(t_{1}, \ldots , t_{q}\right.$ ) means that one takes the limit of $f(s)$ as $s \rightarrow t$ over $s \neq t$ such that $s_{k} \geqslant t_{k}, k = 1, \ldots , q$; the limit from below is defined similarly). By means of specially defined metrics, these spaces can be turned into Polish spaces; an analogue of the modulus of continuity is also introduced in them.

Consider the following model of a randomly walking particle. Let $Y_{1}^{(a)}, Y_{2}^{(a)}, \ldots$ be independent identically distributed random variables, and

Let $\xi_{1}^{(\lambda )}, \xi_{2}^{(\lambda )}, \ldots$ be independent random variables exponentially distributed with parameter $\lambda > 0$, where the sequences $\left\{ Y_{n}^{(a)}\right\}$ and $\left\{ \xi_{m}^{(\lambda )}\right\}$ are independent for all $a$ and $\lambda$. Suppose that the state of the particle changes by $a$ or $-a$ at the times $\tau_{k}^{(\lambda )} = \sum_{j = 1}^{k} \xi_{j}^{(\lambda )}, k = 1,2, \ldots$ (at time $t = 0$ the particle is at zero). More precisely, let the coordinate of the particle at time $\tau_{k}^{(\lambda )}$ be equal to $\sum_{j = 1}^{k} Y_{j}^{(a)}$, and let this state persist on the interval $\left[\tau_{k}^{(\lambda )}, \tau_{k+1}^{(\lambda )}\right)$; thus, the position of the particle at time $t$ is described by the process $X_{t}^{(a, \lambda )} = \sum_{j \leqslant N_{\lambda }(t)} Y_{j}^{(a)}, t \geqslant 0$, where $N_{\lambda }(t) = \max \left\{ k: \tau_{k}^{(\lambda )} \leqslant t\right\}$ (we set $\left.N_{\lambda }(0) = 0, X_{0}^{(a, \lambda )} = 0\right)$.

Investigate the convergence of the finite-dimensional distributions of the processes $X^{(a, \lambda )}$ as $a \rightarrow 0$ and $\lambda \rightarrow \infty$ in such a way that $a^{2} \lambda = \sigma^{2} > 0$ (i.e., the size of the displacement decreases and the collisions become more frequent). Do the processes $X^{(a, \lambda )}$ converge weakly in the space $D[0, \infty )$?

The integral of a function $f$ with respect to a measure $\mathbb {Q}$ (when it is defined) will also be denoted by $\left\langle f, \mathbb {Q} \right\rangle$.

Let $\mathrm{S}$ be a Polish space. Denote by $\mathbf{B}(\mathrm{S}, \mathbb {R})$ the space consisting of bounded $\mathscr {A} \mid \mathscr {B}(\mathbb {R})$ — measurable functions $f: \mathrm{S} \rightarrow \mathbb {R}$, equipped with the norm $\left\| \cdot \right\|_{\infty }$. Let $\mathscr {P} = \mathscr {P}(\mathrm{S})$ be the space of probability measures on $(\mathrm{S}, \mathscr {A})$. Define the variation distance $\left\| \cdot \right\|$ between measures $\mathbb {P}, \mathbb {Q} \in \mathscr {P}$ by the formula

Verify that $\nu (\mathbb {P}, \mathbb {Q}) \equiv \left\| \mathbb {P}-\mathbb {Q}\right\|$ is a distance on $\mathscr {P}$ that metrizes the topology of uniform convergence on $\mathscr {P}$ given by the system of neighborhoods $U(\mathbb {Q}, \delta ) = \left\{ \mathbb {P} \in \mathscr {P}: \nu (\mathbb {P}, \mathbb {Q}) < \delta \right\}$, where $\mathbb {Q} \in \mathscr {P}, \delta > 0$ (on topological spaces, see $[35$; Ch. $2, \S 5])$.

The integral of a function $f$ with respect to a measure $\mathbb {Q}$ (when it is defined) will also be denoted by $\left\langle f, \mathbb {Q} \right\rangle$.

Let $\mathrm{S}$ be a Polish space. Denote by $\mathbf{B}(\mathrm{S}, \mathbb {R})$ the space consisting of bounded $\mathscr {A} \mid \mathscr {B}(\mathbb {R})$ — measurable functions $f: \mathrm{S} \rightarrow \mathbb {R}$, equipped with the norm $\left\| \cdot \right\|_{\infty }$. Let $\mathscr {P} = \mathscr {P}(\mathrm{S})$ be the space of probability measures on $(\mathrm{S}, \mathscr {A})$. Define the variation distance $\left\| \cdot \right\|$ between measures $\mathbb {P}, \mathbb {Q} \in \mathscr {P}$ by the formula

Prove that the following formula holds for the variation distance: $\left\| \mathbb {P}-\mathbb {Q}\right\| = 2 \sup_{A \in \mathscr {A}}\left|\mathbb {P}\left(A\right)-\mathbb {Q}\left(A\right)\right|$.

The integral of a function $f$ with respect to a measure $\mathbb {Q}$ (when it is defined) will also be denoted by $\left\langle f, \mathbb {Q} \right\rangle$.

Let $\mathrm{S}$ be a Polish space. Denote by $\mathbf{B}(\mathrm{S}, \mathbb {R})$ the space consisting of bounded $\mathscr {A} \mid \mathscr {B}(\mathbb {R})$ — measurable functions $f: \mathrm{S} \rightarrow \mathbb {R}$, equipped with the norm $\left\| \cdot \right\|_{\infty }$. Let $\mathscr {P} = \mathscr {P}(\mathrm{S})$ be the space of probability measures on $(\mathrm{S}, \mathscr {A})$. Define the variation distance $\left\| \cdot \right\|$ between measures $\mathbb {P}, \mathbb {Q} \in \mathscr {P}$ by the formula

Set $\nu (\mathbb {P}, \mathbb {Q}) \equiv \left\| \mathbb {P}-\mathbb {Q}\right\|$, as in Exercise 18.

Let measures $\mathbb {P}, \mathbb {Q} \in \mathscr {P}$, and let $\mathbb {P}$ and $\mathbb {Q}$ be absolutely continuous with respect to some measure $\lambda \in \mathscr {P}$ (such a "dominating" measure exists; it suffices to take $\lambda = (\mathbb {P}+\mathbb {Q}) / 2)$. Denote by $p = d \mathbb {P} / d \lambda , q = d \mathbb {Q} / d \lambda$ the Radon-Nikodym derivatives of the measures $\mathbb {P}$ and $\mathbb {Q}$ with respect to the measure $\lambda$. Prove that $\left\| \mathbb {P}-\mathbb {Q}\right\| = \left\| p-q\right\|_{L^{1}(\lambda )}$, where $L^{1}(\lambda ) = L^{1}(\mathrm{S}, \mathscr {A}, \lambda )$. Show that $\nu (\mathbb {P}, \mathbb {Q}) \leqslant 2$, with equality attained only if $\mathbb {P}-\mathbb {Q}$ (the measures are singular, i.e., there exists $A \in \mathscr {A}$ such that $\mathbb {P}\left(A\right) = 1$ and $\mathbb {Q}\left(A\right) = 0)$.

The integral of a function $f$ with respect to a measure $\mathbb {Q}$ (when it is defined) will also be denoted by $\left\langle f, \mathbb {Q} \right\rangle$.

Let $\mathrm{S}$ be a Polish space. Denote by $\mathbf{B}(\mathrm{S}, \mathbb {R})$ the space consisting of bounded $\mathscr {A} \mid \mathscr {B}(\mathbb {R})$ — measurable functions $f: \mathrm{S} \rightarrow \mathbb {R}$, equipped with the norm $\left\| \cdot \right\|_{\infty }$. Let $\mathscr {P} = \mathscr {P}(\mathrm{S})$ be the space of probability measures on $(\mathrm{S}, \mathscr {A})$. Define the variation distance $\left\| \cdot \right\|$ between measures $\mathbb {P}, \mathbb {Q} \in \mathscr {P}$ by the formula

Set $\nu (\mathbb {P}, \mathbb {Q}) \equiv \left\| \mathbb {P}-\mathbb {Q}\right\|$, as in Exercise 18.

Prove that ( $\mathscr {P}(\mathrm{S}), \nu )$ is a complete metric space and that this space is nonseparable if S is uncountable. What can be said about the separability of $(\mathscr {P}(\mathrm{S}), \nu )$ if $\mathrm{S}$ is finite or countable?

Let $\mathrm{S}$ be a Polish space. Let $\mathscr {P} = \mathscr {P}(\mathrm{S})$ be the space of probability measures on $(\mathrm{S}, \mathscr {A})$.

Closely related to the variation distance is the Kakutani-Hellinger distance, defined, in the notation of Exercise 20, by the formula

Verify that $d(\mathbb {P}, \mathbb {Q})$ is a metric on $\mathscr {P}(\mathrm{S})$ whose values do not depend on the choice of the dominating measure $\lambda$.

Denote by $\mathscr {P}(\mathrm{S})$ the space of probability measures on $\mathscr {B}(\mathrm{S})$, and set $\left\langle f, \mathbb {P} \right\rangle = \int_{\mathrm{S}} f d \mathbb {P}$ and $B^{\varepsilon } = \left\{ x \in \mathrm{S}: \rho (x, B) < \varepsilon \right\}$ for $B \subset \mathrm{S}$, $\varepsilon > 0$.

Definition 7. The Lévy-Prokhorov metric is the quantity

Consider the Banach space $B L$ of bounded real Lipschitz functions on a Polish space $\mathrm{S}$, equipped with the norm $\left\| f\right\|_{B L} = \left\| f\right\|_{\infty }+L(f)$, see (41).

Prove that the function

defines a metric on $\mathscr {P}(\mathrm{S})$, convergence in which is also equivalent to weak convergence. Moreover, for any $\mathbb {P}, \mathbb {Q} \in \mathscr {P}(\mathrm{S})$ the following inequalities hold

where $h(t) = 2 t^{2} /(t+2), t \geqslant 0$.

Let S be a locally compact Hausdorff space satisfying the second axiom of countability and $M(\mathrm{S})$ the space of locally finite measures $\mu = \mu (\cdot )$ on the Borel $\sigma$-algebra $\mathscr {B}(\mathrm{S})$ (i.e., the space of measures that are finite on bounded sets of the space S), equipped with the topology in which the mappings

are continuous for functions $f$ from the class $C_{K}^{+}$ of continuous nonnegative functions with compact support.

Prove that $M(\mathrm{S})$ is a Polish space. To do this, note that if $f_{1}, f_{2}, \ldots$ is a set dense in $C_{K}^{+}$, then the function

metrizes the topology introduced above. Moreover, $\mathscr {B}(M(\mathrm{S}))$ is generated both by the mappings $\pi_{f}, f \in C_{K}^{+}$ (i.e., it is the smallest $\sigma$-algebra with respect to which all these mappings are measurable) and by the mappings $\pi_{B}: \mu \mapsto \mu (B)$ for $B \in \mathrm{S}_{\mu }$, where $\mu \in M(\mathrm{S}), \mathrm{S}_{\mu } = \left\{ B \in \mathscr {M}: \mu (\partial B) = 0\right\}$, and $\mathscr {M}$ is the collection of bounded subsets of the space $S$.

Let S be a locally compact Hausdorff space satisfying the second axiom of countability and $M(\mathrm{S})$ the space of locally finite measures $\mu = \mu (\cdot )$ on the Borel $\sigma$-algebra $\mathscr {B}(\mathrm{S})$ (i.e., the space of measures that are finite on bounded sets of the space S), equipped with the topology in which the mappings

are continuous for functions $f$ from the class $C_{K}^{+}$ of continuous nonnegative functions with compact support.

Definition 10. Let a probability space $(\Omega , \mathscr {F}, \mathbb {P})$ and the space of locally finite measures $M(\mathrm{S})$ introduced above be given. A random measure is an $\mathscr {F} \mid \mathscr {B}(M(\mathrm{S}))$-measurable mapping $\mu : \Omega \rightarrow M(\mathrm{S})$.

The collection of bounded subsets of the space $\mathrm{S}$ is denoted by $\mathscr {M}$.

Let $\mu , \mu_{1}, \mu_{2}, \ldots$ be random measures on $S$ — a locally compact Hausdorff space satisfying the second axiom of countability. Prove that the following three conditions are equivalent:

1. $\mu_{n} \stackrel{\mathscr {D}}{\rightarrow } \mu$ as $n \rightarrow \infty$,

2. $\left\langle f, \mu_{n} \right\rangle \stackrel{\mathscr {D}}{\rightarrow } \left\langle f, \mu \right\rangle$ for $f \in C_{K}^{+}(n \rightarrow \infty )$,

3. $\left(\mu_{n}\left(B_{1}\right), \ldots , \mu_{n}\left(B_{k}\right)\right) \stackrel{\mathscr {D}}{\rightarrow }\left(\mu \left(B_{1}\right), \ldots , \mu \left(B_{k}\right)\right)$ for $B_{1}, \ldots , B_{k} \in \mathrm{S}_{\mu }, k \in \mathbb {N}$, where $\mathrm{S}_{\mu } = \left\{ B \in \mathscr {M}: \mu (\partial B) = 0 \text{ a.s.}\right\}$.