Conditioning

Suppose $g_{1}$ and $g_{2}$ are maps from $(X, \mathcal{A})$ to ($y, \mathcal{B}$), with product-measurable graphs. Define $\psi_{i}(x) = \left(x, g_{i}(x)\right)$. Let $P_{i} = \psi_{i}\left(\mu_{i}\right)$, for probability measures $\mu_{i}$ on $\mathcal{A}$. Let $P = \alpha_{1} P_{1}+\alpha_{2} P_{2}$, for constants $\alpha_{i} > 0$ with $\alpha_{1}+\alpha_{2} = 1$. Let $X$ denote the coordinate map onto the $X$ space. Show that the conditional probability distribution $P_{x}$ concentrates on the points $\left(x, g_{1}(x)\right)$ and $\left(x, g_{2}(x)\right)$. Find the conditional probabilities assigned to each point.

Let $Q$ be a nonatomic probability measure on $\mathcal{B}(\mathbb {R})$ with corresponding distribution function $F$. Let $\mathbb {Q} = Q^{n}$, the joint distribution of $n$ independent observations $x_{1}, \ldots , x_{n}$ from $Q$. Define $T\left(x_{1}, \ldots , x_{n}\right): = \max_{i \leq n} x_{i}$.

Show that a $(T, \mu ))$ disintegration $\Lambda = \left\{ \lambda_{t}: t \in \mathcal{T}\right\}$ of a sigma-finite measure $\lambda$ can be chosen as a probability kemel if and only if the sigma-finite measure $\mu$ is equal to the image measure $T \lambda$.

For families of probability measures $\mathcal{P}$ and $Q$ defined on the same sigma-field, define $\alpha_{2}(\mathcal{P}, Q)$ to equal $\sup \left\{ \alpha_{2}(P, Q): P \in \mathcal{P}, Q \in \mathcal{Q}\right\}$, where $\alpha_{2}$ denotes the Hellinger affinity, as in Section 3.3. Write $\mathcal{P} \otimes \mathcal{P}$ for $\left\{ P_{1} \otimes P_{2}: P_{i} \in \mathcal{P}\right\}$, and $\operatorname {co}(\mathcal{P} \otimes \mathcal{P})$ for its convex hull. Show that $\alpha_{2}(\operatorname {co}(\mathcal{P} \otimes \mathcal{P}), \operatorname {co}(\mathcal{Q} \otimes \mathbb {Q})) \leq \alpha_{2}(\operatorname {co}(\mathcal{P}), \operatorname {co}(\mathcal{Q}))^{2}$ by the following steps. Write $A$ for $\alpha_{2}(\operatorname {co}(\mathcal{P}), \operatorname {co}(Q))$.

Let $\mathbb {P}$ be Lebesgue measure on $\mathcal{F}$, the Borel sigma-field of $[0,1]$. Let $\mathcal{G}$ denote the sigma-field generated by all the singletons in $[0,1]$.

Let $X$ be an integrable random variable, and $Z$ be a $\mathcal{G}$-measurable random variable for which $X Z$ is integrable. Show that $\mathbb {E}\left[X Z\right] = \mathbb {E}\left[Y Z\right]$ where $Y = \mathbb {E}\left[X \mid \mathcal{G}\right]$.

Suppose $X_{n} \in \mathcal{M}^{+}(\mathcal{F})$. Show that $\mathbb {E}\left[\liminf_{n} X_{n} \mid \mathcal{G}\right] \leq \liminf_{n} \mathbb {E}\left[X_{n} \mid \mathcal{G}\right]$ almost surely, for each sub-sigma-field $\mathcal{G}$.

Suppose $X_{n} \rightarrow X$ almost surely, with $\left|X_{n}\right| \leq H$ for an integrable $H$. Show that $\mathbb {E}\left[X_{n} \mid \mathcal{G}\right] \rightarrow \mathbb {E}\left[X \mid \mathcal{G}\right]$ almost surely.

Suppose $\mathbb {E}\left[X^{2}\right] < \infty$. Show that $\operatorname {Var}\left[X\right] = \operatorname {Var}\left[\mathbb {E}\left[X \mid \mathcal{G}\right]\right]+\mathbb {E}\left[\operatorname {Var}\left[X \mid \mathcal{G}\right]\right]$.

Let $X$ be an integrable random variable on a probability space $(\Omega , \mathcal{F}, \mathbb {P})$. Let $\mathcal{G}$ be a sub-sigma-field of $\mathcal{F}$ containing all $\mathbb {P}$-negligible sets. Show that $X$ is $\mathcal{G}$-measurable if and only if $\mathbb {E}\left[X W\right] = 0$ for every bounded random variable $W$ with $\mathbb {E}\left[W \mid \mathcal{G}\right] = 0$ almost surely. (Compare with the corresponding statement for random variables that are square integrable: $Z \in \mathcal{L}^{2}(\mathcal{G})$ if and only if it is orthogonal to every square integrable $W$ that is orthogonal to $\mathcal{L}^{2}(\mathcal{G})$.)

Let $S_{0}, S_{1}, \ldots , S_{N}$ be random vectors, and $\mathcal{F}_{0} \subseteq \mathcal{F}_{1} \subseteq \ldots \subseteq \mathcal{F}_{N} \subseteq \mathcal{F}$ be sigma-fields such that $S_{i}$ is $\mathcal{F}_{i}$-measurable for each $i$. Let $\epsilon , \alpha$, and $\beta$ be positive numbers for which $\beta \mathbb {P}\left(\left|S_{N}-S_{i}\right| \leq (1-\alpha )\left|S_{i}\right| \mid \mathcal{F}_{i}\right) \geq \left\{ \left|S_{i}\right| > \epsilon \right\}$ almost surely. Show that $\mathbb {P}\left(\max_{i}\left|S_{i}\right| > \epsilon \right) \leq \beta \mathbb {P}\left(\left|S_{N}\right| > \alpha \epsilon \right)$.

Suppose $P$ is a probability measure on $(X, \mathcal{A})$ with density $p(x)$ with respect to a probability measure $\lambda$. Let $Q$ equal the image $T P$, and $\mu$ equal $T \lambda$, for a measurable map $T$ into ($\mathcal{T}, \mathcal{B}$). Without assuming existence of disintegrations or conditional distributions, show

For each convex function $\psi$ on the real line there exists a countable family of linear functions for which $\psi (x) = \sup_{i \in \mathbb {N}}\left(a_{i}+b_{i} x\right)$ for all $x$ (see Appendix C). Use this representation to prove the conditional form of Jensen's inequality: if $X$ and $\psi (X)$ are both integrable then $\mathbb {E}\left[\psi (X) \mid \mathcal{G}\right] \geq \psi \left(\mathbb {E}\left[X \mid \mathcal{G}\right]\right)$ almost surely.

Let $P$ be a probability measure on $\mathcal{B}\left(\mathbb {R}^{2}\right)$ that is invariant under rotations $S_{\theta }$ for a dense set $\Theta_{0}$ of angles. Show that it is invariant under all rotations.

Find an invariance argument that produces the conditional distributions from Exercise $\langle 5 \rangle$.

Exercise 5: Let $\mathbb {P}$ denote Lebesgue measure on $\mathcal{B}\left([0,1]^{2}\right)$. Let $T(x, y) = \max (x, y)$. The conditional probability distributions $\left\{ \mathbb {P}_{t}\right\}$ for $\mathbb {P}$ given $T$ are uniform on the sets $\left\{ T = t\right\}$.

Let $Q$ be a family of probability measures on a sigma-field $\mathcal{A}$ dominated by a fixed sigma-finite measure $\lambda$. That is, each $Q$ in $Q$ has a density $\delta_{Q}$ with respect to $\lambda$. Show that $Q$ is also dominated by a probability measure of the form $v = \sum_{j = 1}^{\infty } 2^{-j} Q_{j}$, with $Q_{j} \in Q$ for each $j$, by following these steps.

Let $\mathcal{P}_{\Theta } = \left\{ \mathbb {P}_{\theta }: \theta \in \Theta \right\}$ be a family of probability measures on $\mathcal{F}$, dominated by a sigma-finite measure $\lambda$. Suppose $\mathcal{G}_{0}$ is a weakly sufficient sub-sigma-field, and $\mathcal{G}_{0} \subseteq \mathcal{G}_{1}$ for another sigma-field $\mathcal{G}_{1} \subseteq \mathcal{F}$. Show that $\mathcal{G}_{1}$ is also weakly sufficient, by following these steps.

Let $\mathcal{P}_{\Theta } = \left\{ \mathbb {P}_{\theta }: \theta \in \Theta \right\}$ be a family of probability measures dominated by a sigma-finite measure $\lambda$. Suppose $T$ is a strongly sufficient statistic for $\mathcal{P}$, meaning that there exists a probability kernel $\left\{ P_{t}: t \in \mathcal{T}\right\}$ that is a conditional distribution for each $\mathbb {P}_{\theta }$ given $T$. Show that there exist versions of the densities $d \mathbb {P}_{\theta } / d \lambda$ of the form $g(T \omega , \theta ) h(\omega )$, by following these steps.

Let $\mathcal{P}_{\Theta } = \left\{ \mathbb {P}_{\theta }: \theta \in \Theta \right\}$ be a family of probability measures dominated by a sigma-finite measure $\lambda$. Suppose $T$ is a strongly sufficient statistic for $\mathcal{P}$. Suppose $T^{*}$ is another statistic, taking values in a set $\mathcal{T}^{*}$ equipped with a countably generated sigma-field $\mathcal{B}^{*}$ containing the singletons, such that $\sigma (T) \subseteq \sigma \left(T^{*}\right)$. Show that $T$ can be written as a measurable function of $T^{*}$. Deduce via the factorization criterion that $T^{*}$ is also strongly sufficient.

Let $(X, \mathcal{A}, P)$ be a probability space, and let $\mathbb {P}$ equal $P^{n}$, the $n$-fold product measure on $\mathcal{A}^{\mathbb {N}}$. For each $x$ in $X$, let $\delta (x)$ denote the point mass at $x$. For $\mathbf{x}$ in $X^{n}$, let $T(\mathbf{x})$ denote the so-called empirical measure, $n^{-1} \sum_{i \leq n} \delta \left(x_{i}\right)$. Intuitively, if we are given the measure $T(\mathbf{x})$, the conditional distribution for $\mathbb {P}$ should give mass $1 / n$ ! to each of the $n$ ! permutations $\left(x_{1}, \ldots , x_{n}\right)$. Formalize this intuition by constructing a ($T, T \mathbb {P}$)-disintegration for $\mathbb {P}$. Warning: Not as easy as it seems. What is the $(\mathcal{T}, \mathcal{B})$ for this problem? What happens if the empirical measure is not supported by $n$ distinct points? How should you define $\mathbb {P}_{t}$ when $t$ does not correspond to a possible realization of an empirical measure?