Math Problems Vault

Problem 7.9.1

(a)(b)

Let $Y$ be uniformly distributed on $[-1,1]$ and let $X = Y^{2}$ .

?

(a)

Find the best predictor of $X$ given $Y$ , and of $Y$ given $X$ .

(b)

Find the best linear predictor of $X$ given $Y$ , and of $Y$ given $X$ .

Problem 7.9.2

(a)(b)

?

(a)

Let the pair $(X, Y)$ have a general bivariate normal distribution. Find $\mathbb {E}\left[Y \mid X\right]$ .

(b)

Let $U_{1}, U_{2}, \ldots , U_{n}$ be independent $N(0,1)$ random variables, and let $\left(a_{i}\right),\left(b_{i}\right)$ be real non-zero vectors. Show that $X = \sum_{i} a_{i} U_{i}$ and $Y = \sum_{i} b_{i} U_{i}$ satisfy

\mathbb {E}\left[Y \mid X\right] = X \frac{\sum _{i} a_{i} b_{i}}{\sum _{i} a_{i}^{2}}

Problem 7.9.3

Let $X_{1}, X_{2}, \ldots , X_{n}$ be random variables with zero means and covariance matrix $\mathbf{V} = \left(v_{i j}\right)$ , and let $Y$ have finite second moment. Find the linear function $h$ of the $X_{i}$ which minimizes the mean squared error $\mathbb {E}\left[\left(Y-h\left(X_{1}, \ldots , X_{n}\right)\right)^{2}\right]$ .

?

Problem 7.9.4

Verify the following properties of conditional expectation. You may assume that the relevant expectations exist.

?

(a)

$\mathbb {E}\left[\mathbb {E}\left[Y \mid g\right]\right] = \mathbb {E}\left[Y\right]$ .

(b)

$\mathbb {E}\left[\alpha Y+\beta Z \mid g\right] = \alpha \mathbb {E}\left[Y \mid g\right] + \beta \mathbb {E}\left[Z \mid g\right]$ for $\alpha , \beta \in \mathbb {R}$ .

(c)

$\mathbb {E}\left[Y \mid g\right] \geq 0$ if $Y \geq 0$ .

(d)

$\mathbb {E}\left[Y \mid g\right] = \mathbb {E}\left[\mathbb {E}\left[Y \mid \mathcal{H}\right] \mid g\right]$ if $\mathcal{G} \subseteq \mathcal{H}$ .

(e)

$\mathbb {E}\left[Y \mid \mathcal{G}\right] = \mathbb {E}\left[Y\right]$ if $Y$ is independent of $I_{G}$ for every $G \in \mathcal{G}$ .

(f)

Jensen's inequality. $g\left\{ \mathbb {E}\left[Y \mid g\right]\right\} \leq \mathbb {E}\left[g(Y) \mid g\right]$ for all convex functions $g$ .

(g)

If $Y_{n} \xrightarrow {\text{ a.s. }} Y$ and $\left|Y_{n}\right| \leq Z$ a.s. where $\mathbb {E}\left[Z\right] < \infty$ , then $\mathbb {E}\left[Y_{n} \mid \mathcal{g}\right] \xrightarrow {\text{ a.s. }} \mathbb {E}\left[Y \mid \mathcal{g}\right]$ .

Statements (b)-(f) are of course to be interpreted 'almost surely'.

Problem 7.9.5

Let $X$ and $Y$ have joint mass function $f(x, y) = \left\{ x(x+1)\right\}^{-1}$ for $x = y = 1,2, \ldots$ . Show that $\mathbb {E}\left[Y \mid X\right] < \infty$ while $\mathbb {E}\left[Y\right] = \infty$

?

Problem 7.9.6

(a)(b)

Let $(\Omega , \mathcal{F}, \mathbb {P})$ be a probability space and let $\mathcal{G}$ be a sub- $\sigma$ -field of $\mathcal{F}$ . Let $H$ be the space of $\mathcal{G}$ -measurable random variables with finite second moment.

?

(a)

Show that $H$ is closed with respect to the norm $\left\| \cdot \right\|_{2}$ .

(b)

Let $Y$ be a random variable satisfying $\mathbb {E}\left[Y^{2}\right] < \infty$ , and show the equivalence of the following two statements for any $M \in H$ :

(i) $\mathbb {E}\left[(Y-M) Z\right] = 0$ for all $Z \in H$ ,

(ii) $\mathbb {E}\left[(Y-M) I_{G}\right] = 0$ for all $G \in \mathcal{G}$ .

Problem 7.9.7

For possibly dependent random variables $X$ and $Y$ , define the maximal correlation coefficient $m(X, Y) = \sup \rho (f(X), g(Y))$ , where $\rho$ denotes ordinary correlation, and the supremum is over all functions $f$ and $g$ such that $f(X)$ and $g(Y)$ have finite non-zero variances. Show that:

?

(a)

$m(X, Y) = 0$ if and only if $X$ and $Y$ are independent,

(b)

$m(X, Y)^{2} = \sup_{g} \operatorname {Var}\left[\mathbb {E}\left[g(Y) \mid X\right]\right]$ , where the supremum is over all functions $g$ such that $\operatorname {Var}\left[g(Y)\right] = 1$

(c)

$\widehat{f}(X) m(X, Y) = \mathbb {E}\left[\widehat{g}(Y) \mid X\right]$ a.s., where $\widehat{f}$ and $\widehat{g}$ are functions such that $m = \rho (\widehat{f}(X), \widehat{g}(Y))$ .

(d)

We have, a.s., that

\mathbb {E}\left[\mathbb {E}\left[\widehat{f}(X) \mid Y\right] \mid X\right] = m(X, Y)^{2} \widehat{f}(X), \quad \mathbb {E}\left[\mathbb {E}\left[\widehat{g}(Y) \mid X\right] \mid Y\right] = m(X, Y)^{2} \widehat{g}(Y)

(e)

If the ordered triple $X, Y, Z$ is a Markov chain, show that $m(X, Z) \leq m(X, Y) m(Y, Z)$ , with equality if $(X, Y)$ and $(Z, Y)$ are identically distributed.

(f)

Deduce that, for a pair $(U, V)$ with the standard bivariate normal distribution, $m(U, V)$ is an increasing function of the modulus $\left|\rho \right|$ of correlation. [It can be shown that $m(U, V) = \left|\rho \right|$ in this case.]

Problem 7.9.8

Let $\rho_{\operatorname {mon}}(X, Y) = \sup \rho (f(X), g(Y))$ , where the supremum is over all monotonic functions $f$ and $g$ such that $f(X)$ and $g(Y)$ have finite, non-zero variances. Show that $\rho_{\text{mon }}(X, Y) = 0$ if and only if $X$ and $Y$ are independent.

For random variables $X, Y$ with finite, non-zero variances, show that $\rho (X, Y) \leq \rho_{\operatorname {mon}}(X, Y) \leq m(X, Y)$ , where $m$ is the maximal correlation coefficient.

?

Problem 7.9.9

(a)(b)(c)

Let $(X, Y)$ have joint density function $f(x, y) = 2 e^{-x-y}$ for $0 < x \leq y < \infty$ .

?

(a)

Find the minimum mean-squared-error predictor of $X$ given that $Y = y$ .

(b)

Find the minimum mean-squared-error linear predictor of $X$ given that $Y = y$ .

(c)

Compare these.