Conditional Probability

20.27 ↑ Borel's paradox. Suppose that a random point on the sphere is specified by longitude $\Theta$ and latitude $\Phi$, but restrict $\Theta$ by $0 \leq \Theta <\pi$, so that $\Theta$ specifies the complete meridian circle (not semicircle) containing the point, and compensate by letting $\Phi$ range over ( $-\pi , \pi$ ].

$20.16 \uparrow$ Let $X$ and $Y$ be independent, each having the standard normal distribution, and let $(R, \Theta )$ be the polar coordinates for $(X, Y)$.

↑ Paradoxes of a somewhat similar kind arise in very simple cases.

$\uparrow$

$\uparrow$ Suppose that $T=X$ is a random variable, $\left(\Omega^{\prime }, \mathscr {F}^{\prime }\right)=\left(R^{1}, \mathscr {R}^{1}\right)$, and $x$ is the general point of $R^{1}$. In this case $p(A \mid x)$ is sometimes written $\mathbb {P}\left( A \mid X=x \right)$. What is the problem with this notation?

For the Poisson process (see Example 33.1) show that for $0<s<t$,

Thus the conditional distribution (in the sense of Theorem 33.3) of $N_{s}$ given $N_{t}$ is binomial with parameters $N_{t}$ and $s / t$.

$29.12 \uparrow$ Suppose that $\left(X_{1}, X_{2}\right)$ has the centered normal distribution-has in the plane the distribution with density (29.10). Express the quadratic form in the exponential as

integrate out the $x_{2}$ and show that

where $\tau =\sigma_{22}-\sigma_{12}^{2} \sigma_{11}^{-1}$. Describe the conditional distribution of $X_{2}$ given $X_{1}$.

↑ Deduce from (33.16) that the conditional distribution of $X$ given $M$ is

where $\mu$ is the distribution corresponding to $F$ (positive and continuous). Hint: First check $H=(-\infty , x]$.

$4.1012 .4 \uparrow$ The following construction shows that conditional probabilities may not give measures. Complete the details.

In Problem 4.10 it is shown that there exist a probability space $(\Omega , \mathscr {F}, P)$, a $\sigma$-field $\mathscr {G}$ in $\mathscr {F}$, and a set $H$ in $\mathscr {F}$ such that $\mathbb {P}\left(H\right)=\frac{1}{2}, H$ and $\mathscr {G}$ are independent, $\mathscr {G}$ contains all the singletons, and $\mathscr {G}$ is generated by a countable subclass. The countable subclass generating $\mathscr {G}$ can be taken to be a $\pi$-system $\mathscr {P}=\left\{ B_{1}, B_{2}, \ldots \right\}$ (pass to the finite intersections of the sets in the original class).

Assume that it is possible to choose versions $P[A \| \mathscr {G}]$ so that $P[A \| \mathscr {G}]_{\omega }$ is for each $\omega$ a probability measure as $A$ varies over $\mathscr {F}$. Let $C_{n}$ be the $\omega$-set where $P\left[B_{n} \| \mathscr {G}\right]_{\omega }=I_{B_{n}}(\omega )$; show (Example 33.3) that $C=\bigcap_{n} C_{n}$ has probability 1. Show that $\omega \in C$ implies that $P[G \| \mathscr {A}]_{\omega }=I_{G}(\omega )$ for all $G$ in $\mathscr {I}$ and hence that $P[\left\{ \omega \right\} \| \mathscr {G}]_{\omega }=1$.

Now $\omega \in H \cap C$ implies that $P[H \| \mathscr {G}]_{\omega } \geq P[\left\{ \omega \right\} \| \mathscr {G}]_{\omega }=1$ and $\omega \in H^{c} \cap C$ implies that $P[H \| \mathscr {G}]_{\omega } \leq P[\Omega -\left\{ \omega \right\} \| \mathscr {G}]_{\omega }=0$. Thus $\omega \in C$ implies that $P[H \| \mathscr {G}]_{\omega }=I_{H}(\omega )$. But since $H$ and $\mathscr {G}$ are independent, $P[H \| \mathscr {G}]_{\omega }= \mathbb {P}\left(H\right)=\frac{1}{2}$ with probability 1, a contradiction.

This example is related to Example 4.10 but concerns mathematical fact instead of heuristic interpretation.

Let $\alpha$ and $\beta$ be $\sigma$-finite measures on the line, and let $f(x, y)$ be a probability density with respect to $\alpha \times \beta$. Define

unless the denominator vanishes, in which case take $g_{x}(y)=0$, say. Show that, if $(X, Y)$ has density $f$ with respect to $\alpha \times \beta$, then the conditional distribution of $Y$ given $X$ has density $g_{X}(y)$ with respect to $\beta$. This generalizes Examples 33.5 and 33.12, where $\alpha$ and $\beta$ are Lebesgue measure.

$18.20 \uparrow$ Suppose that $\mu$ and $\nu_{x}$ (one for each real $x$ ) are probability measures on the line, and suppose that $\nu_{x}(B)$ is a Borel function in $x$ for each $B \in \mathscr {R}^{1}$. Then (see Problem 18 20)

defines a probability measure on $(R^{2}, \mathscr {R}^{2})$. Suppose that $(X, Y)$ has distribution $\pi$, and show that $\nu_{X}$ is a version of the conditional distribution of $Y$ given $X$.

$\uparrow$ Let $\alpha$ and $\beta$ be $\sigma$-finite measures on the line. Specialize the setup of Problem 33.13 by supposing that $\mu$ has density $f(x)$ with respect to $\alpha$ and $\nu$ has density $g_{x}(y)$ with respect to $\beta$. Assume that $g_{x}(y)$ is measurable $\mathscr {R}^{2}$ in the pair $(x, y)$, so that $\nu_{x}(B)$ is automatically measurable in $x$. Show that (33.35) has density $f(x) g_{x}(y)$ with respect to $\alpha \times \beta : \pi (E)= \iint_{E} f(x) g_{x}(y) \alpha (d x) \beta (d y)$. Show that (33.34) is consistent with $f(x, y)= f(x) g_{x}(y)$. Put

Suppose that $(X, Y)$ has density $f(x) g_{x}(y)$ with respect to $\alpha \times \beta$, and show that $p_{Y}(x)$ is a density with respect to $\alpha$ for the conditional distribution of $X$ given $Y$.

In the language of Bayes, $f(x)$ is the prior density of a parameter $x, g_{x}(y)$ is the conditional density of the observation $y$ given the parameter, and $p_{y}(x)$ is the posterior density of the parameter given the observation.

$\uparrow$ Now suppose that $\alpha$ and $\beta$ are Lebesgue measure, that $f(x)$ is positive, continuous, and bounded, and that $g_{x}(y)=e^{-(y-x)^{2} n / 2} / \sqrt{2 \pi / n}$. Thus the observation is distributed as the average of $n$ independent normal variables with mean $x$ and variance 1. Show that

for fixed $x$ and $y$. Thus the posterior density is approximately that of a normal distribution with mean $y$ and variance $1 / n$.

$32.13 \uparrow$ Suppose that $X$ has distribution $\mu$. Now $P[A \| X]_{\omega }=f(X(\omega ))$ for some Borel function $f$. Show that $\lim_{h \rightarrow 0} \mathbb {P}\left( A \mid x-h<X \leq x+h \right)=f(x)$ for $x$ in a set of $\mu$-measure 1. Roughly speaking, $\mathbb {P}\left( A \mid x-h<X \leq x+h \right) \rightarrow \mathbb {P}\left( A \mid X =x \right)$. Hint: Take $\nu (B)=\mathbb {P}\left(A \cap [X \in B]\right)$ in Problem 32.13.