Chapter 12

Multivariate normal distributions

[8/0%]
Show
LaTeX
Problem 12.1

Let V0V_{0} and V1V_{1} be positive definition matrices. For 0≤θ≤10 \leq \theta \leq 1 let gθg_{\theta } denote the N(0,(1−θ)V0+θV1)N\left(0,(1-\theta ) V_{0}+\theta V_{1}\right) density on Rn\mathbb {R}^{n}. Show that

∂gθ∂θ=12∑j,kΔj,k∂2gθ∂zj∂zk where Δ:=V1−V0 \frac{\partial g_{\theta }}{\partial \theta } = \frac{1}{2} \sum _{j, k} \Delta _{j, k} \frac{\partial ^{2} g_{\theta }}{\partial z_{j} \partial z_{k}} \quad \text{ where } \Delta : = V_{1}-V_{0}

by following these steps.

?
(i)

Use Fourier inversion to show that gθ(x)=(2π)−n∫exp⁡(−ix′t−12t′Vθt)dtg_{\theta }(x) = (2 \pi )^{-n} \int \exp \left(-i x^{\prime } t-\frac{1}{2} t^{\prime } V_{\theta } t\right) d t.

(ii)

Justify differentiation under the integral sign to show that

∂gθ(x)∂θ=(2π)−n∫∂∂θexp⁡(−ix′t−12t′(θΔ+V0)t)dt=(2π)−n∫−12t′Δtexp⁡(−ix′t−12t′(θΔ+V0)t)dt \begin{aligned} \frac{\partial g_{\theta }(x)}{\partial \theta } & = (2 \pi )^{-n} \int \frac{\partial }{\partial \theta } \exp \left(-i x^{\prime } t-\frac{1}{2} t^{\prime }\left(\theta \Delta +V_{0}\right) t\right) d t \\ & = (2 \pi )^{-n} \int -\frac{1}{2} t^{\prime } \Delta t \exp \left(-i x^{\prime } t-\frac{1}{2} t^{\prime }\left(\theta \Delta +V_{0}\right) t\right) d t \end{aligned}

and

∂2gθ(x)∂xj∂xk=(2π)−n∫∂2∂xj∂xkexp⁡(−ix′t−12t′(θΔ+V0)t)dt=(2π)−n∫(−itj)(−itk)exp⁡(−ix′t−12t′(θΔ+V0)t)dt. \begin{aligned} \frac{\partial ^{2} g_{\theta }(x)}{\partial x_{j} \partial x_{k}} & = (2 \pi )^{-n} \int \frac{\partial ^{2}}{\partial x_{j} \partial x_{k}} \exp \left(-i x^{\prime } t-\frac{1}{2} t^{\prime }\left(\theta \Delta +V_{0}\right) t\right) d t \\ & = (2 \pi )^{-n} \int \left(-i t_{j}\right)\left(-i t_{k}\right) \exp \left(-i x^{\prime } t-\frac{1}{2} t^{\prime }\left(\theta \Delta +V_{0}\right) t\right) d t . \end{aligned}
(iii)

Collect terms.

Problem 12.2

Suppose XnX_{n} has a multivariate normal distribution, and Xn⇝XX_{n} \leadsto X. Show that XX also has a multivariate normal distribution.

?
Problem 12.3

Let Z1,Z2,…Z_{1}, Z_{2}, \ldots be independent random variables, each distributed N(0,1)N(0,1). Show that the distribution of Mn:=max⁡i≤nZiM_{n}: = \max_{i \leq n} Z_{i} concentrates largely in a range of order (log⁡n)−1/2(\log n)^{-1 / 2} by following these steps. Define αn:=(2log⁡n)1/2\alpha_{n}: = (2 \log n)^{1 / 2} and Ln=12log⁡log⁡nL_{n} = \frac{1}{2} \log \log n.

?
(i)

Remember (Appendix D) that the tail function Φˉ(x):=P(N(0,1)>x)\bar{\Phi }(x): = \mathbb {P}\left(N(0,1) > x\right) decreases like ϕ(x)/x\phi (x) / x as x→∞x \rightarrow \infty, in the sense that the ratio of the two functions tends to 1. For a fixed constant CC, define xn:=αn−(C+Ln)/αnx_{n}: = \alpha_{n}-\left(C+L_{n}\right) / \alpha_{n}. Show that log⁡(nΦˉ(xn))\log \left(n \bar{\Phi }\left(x_{n}\right)\right) converges to C0:=C−12log⁡(4π)C_{0}: = C-\frac{1}{2} \log (4 \pi ) as n→∞n \rightarrow \infty.

(ii)

Deduce that log⁡P(Mn≤xn)=nlog⁡(1−Φˉ(xn))→e−C0\log \mathbb {P}\left(M_{n} \leq x_{n}\right) = n \log \left(1-\bar{\Phi }\left(x_{n}\right)\right) \rightarrow e^{-C_{0}}.

(iii)

Deduce that αn(Mn−αn)+Ln\alpha_{n}\left(M_{n}-\alpha_{n}\right)+L_{n} converges in distribution as n→∞n \rightarrow \infty.

Problem 12.4

Let XX and YY be random variables for which both P(X>x)≤P(Z>x)\mathbb {P}\left(X > x\right) \leq \mathbb {P}\left(Z > x\right) and P(X≤−x)≤P(Z≤−x)\mathbb {P}\left(X \leq -x\right) \leq \mathbb {P}\left(Z \leq -x\right), for all x≥0x \geq 0. Show that E[exp⁡(tX)]≤E[exp⁡(tY)]\mathbb {E}\left[\exp (t X)\right] \leq \mathbb {E}\left[\exp (t Y)\right] for all nonnegative tt.

?
Problem 12.5

Let AA be a closed subset of R2\mathbb {R}^{2}, with sections Ay:={x∈R:(x,y)∈A}A_{y}: = \left\{ x \in \mathbb {R}:(x, y) \in A\right\} having Gaussian measure g(y):=γ1Ayg(y): = \gamma_{1} A_{y}. Let BB denote the 1-shift, as in Lemma ⟨14⟩\langle 14 \rangle.

Lemma 14: For a closed set A⊆R2A \subseteq \mathbb {R}^{2} with sections Ay:={x∈R:(x,y)∈A}A_{y}: = \left\{ x \in \mathbb {R}:(x, y) \in A\right\}, define h(y)h(y) by Φ(h(y)):=γ1Ay\Phi (h(y)): = \gamma_{1} A_{y} (Φ\Phi the standard normal distribution function), and let B:={(x,y):x≤h(y)}B: = \left\{ (x, y): x \leq h(y)\right\} be the 1-shift of AA --- the set obtained by sliding and stretching each xx-section of AA into a semi-infinite interval with the same γ1\gamma_{1} measure, so that the yy-sections of BB have the same γ1\gamma_{1} measure as the corresponding sections of AA. Then BB is an improvement over AA: γ2B≥γ2A\gamma_{2} B \geq \gamma_{2} A and γ2Br≤γ2Ar\gamma_{2} B^{r} \leq \gamma_{2} A^{r} for every r≥0r \geq 0 (where ArA^{r} denotes the rr-neighborhood of AA).

?
(i)

If yn→yy_{n} \rightarrow y, show that lim⁡sup⁡Ayn≤Ay\lim \sup A_{y_{n}} \leq A_{y}, in the sense of pointwise convergence of indicator functions. (If x∈Aynx \in A_{y_{n}} infinitely often, deduce that (x,y)∈A(x, y) \in A.)

(ii)

If yn→yy_{n} \rightarrow y, use Fatou's lemma to prove that lim sup⁡g(yn)≤g(y)\limsup g\left(y_{n}\right) \leq g(y).

(iii)

If (xn,yn)∈B\left(x_{n}, y_{n}\right) \in B and (xn,yn)→(x,y)\left(x_{n}, y_{n}\right) \rightarrow (x, y), show that x≤g(y)x \leq g(y), that is, (x,y)∈B(x, y) \in B.

Problem 12.6

Reduce the one-dimensional version of Theorem ⟨7⟩\langle 7 \rangle to the case where AA is a finite union of intervals, by following these steps. Let Φ(α):=γ1A\Phi (\alpha ): = \gamma_{1} A.

Theorem 7: The Gaussian measure of the neighborhood ArA^{r} is minimized, for a given value of γkA\gamma_{k} A, by choosing AA as a closed half-space. More generally, γkAr≥Φ(r+a)\gamma_{k} A^{r} \geq \Phi (r+a), for every Borel subset AA of Rk\mathbb {R}^{k} with γkA≥Φ(a)\gamma_{k} A \geq \Phi (a).

?
(i)

Show that it is enough prove that γAr+δ≥Φ(r+α)\gamma A^{r+\delta } \geq \Phi (r+\alpha ) for each δ>0\delta > 0, because Ar+δ↓ArA^{r+\delta } \downarrow A^{r} as δ\delta decreases to zero.

(ii)

Define an open set G:={x:d(x,A)<δ}G: = \left\{ x: d(x, A) < \delta \right\}. Show that γG>α\gamma G > \alpha. Hint: The set G\AˉG \backslash \bar{A} is open.

(iii)

Show that there exists a countable family {Ji}\left\{ J_{i}\right\} of disjoint closed intervals for which γ1(G\∪iJi)=0\gamma_{1}\left(G \backslash \cup_{i} J_{i}\right) = 0.

(iv)

Choose NN so that the closed set AN:=∪i≤NJiA_{N}: = \cup_{i \leq N} J_{i} has γ1\gamma_{1} measure greater than Φ(α)\Phi (\alpha ). Show that the assertion of the Theorem for ANA_{N} implies that γAr+δ≥Φ(r+α)\gamma A^{r+\delta } \geq \Phi (r+\alpha ).

Problem 12.7

(Slepian's (1962) inequality) Let X=(X1,…,Xn)X = \left(X_{1}, \ldots , X_{n}\right) and Y=(Y1,…,Yn)Y = \left(Y_{1}, \ldots , Y_{n}\right) both have centered multivariate normal distributions with Var⁡[Xj]=Var⁡[Yj]\operatorname {Var}\left[X_{j}\right] = \operatorname {Var}\left[Y_{j}\right] for each jj and E[XjXk]≤E[YjYk]\mathbb {E}\left[X_{j} X_{k}\right] \leq \mathbb {E}\left[Y_{j} Y_{k}\right] for all j≠kj \neq k. Prove that P(∪j{Xj>αj})≥P(∪j{Yj>αj})\mathbb {P}\left(\cup_{j}\left\{ X_{j} > \alpha_{j}\right\} \right) \geq \mathbb {P}\left(\cup_{j}\left\{ Y_{j} > \alpha_{j}\right\} \right) for all real numbers α1,…,αn\alpha_{1}, \ldots , \alpha_{n}.

?
Problem 12.8

Let SS be a nonnegative random variable with P(S>M+r)≤C0exp⁡(−12r2/σ2)\mathbb {P}\left(S > M+r\right) \leq C_{0} \exp \left(-\frac{1}{2} r^{2} / \sigma^{2}\right) for all r≥0r \geq 0, for positive constants C0,MC_{0}, M, and σ2\sigma^{2}.

?
(i)

Show that E[exp⁡(αS2)]=1+∫0∞2yαexp⁡(αy2)P(S>y)dy\mathbb {E}\left[\exp \left(\alpha S^{2}\right)\right] = 1+\int_{0}^{\infty } 2 y \alpha \exp \left(\alpha y^{2}\right) \mathbb {P}\left(S > y\right) d y.

(ii)

For α<1/(2σ2)\alpha < 1 /\left(2 \sigma^{2}\right), prove that E[exp⁡(αS2)]<∞\mathbb {E}\left[\exp \left(\alpha S^{2}\right)\right] < \infty.