Gaussian random vectors and processes

Let $X, Y$ be IID rv s, each with density $f_X(x) = \alpha \exp (-x^2/2)$. In (b), we show that $\alpha$ must be $1/\sqrt{2\pi }$ in order for $f_X(x)$ to integrate to 1, but in this part, we leave $\alpha$ undetermined. Let $S = X^2 + Y^2$. Find the probability density of $S$ in terms of $\alpha$.

Prove from (a) that $\alpha$ must be $1/\sqrt{2\pi }$ in order for $S$, and thus $X$ and $Y$, to be rv s. Show that $\mathbb {E}\left[X\right] = 0$ and that $\mathbb {E}\left[X^2\right] = 1$.

Find the probability density of $R = \sqrt{S}$. $R$ is called a Rayleigh rv.

By expanding in a power series in $(1/2)r^2\sigma^2$, show that

By expanding $e^{rZ}$ in a power series in $rZ$, show that

By matching powers of $r$ between (a) and (b), show that for all integer $k \geq 1$,

Let $X_1 \sim N(0,\sigma_1^2)$ and let $X_2 \sim N(0,\sigma_2^2)$ be independent of $X_1$. Convolve the density of $X_1$ with that of $X_2$ to show that $X_1+X_2$ is Gaussian, $N(0,\sigma_1^2+\sigma_2^2)$.

Let $W_1, W_2$ be IID normalized Gaussian rv s. Show that $a_1W_1+a_2W_2$ is Gaussian, $N(0,a_1^2+a_2^2)$.

Combine (b) with induction to show that all linear combinations of IID normalized Gaussian rv s are Gaussian.

Let $\overrightarrow {U}$ be an $n$-rv with mean $\overrightarrow {m}$ and covariance $[K]$ whose MGF is given by (3.18):

Let $X = \overrightarrow {r}^{\mathrm T}\overrightarrow {U}$ for an arbitrary real $n$-vector $\overrightarrow {r}$. Show that the MGF of $X$ is given by $g_X(r) = \exp \left[r\mathbb {E}\left[X\right]+r^2\sigma_X^2/2\right]$ and relate $\mathbb {E}\left[X\right]$ and $\sigma_X^2$ to $\overrightarrow {m}$ and $[K]$.

Show that $\overrightarrow {U}$ is a Gaussian rv.

Let $\overrightarrow {Z} \sim N(0,[K])$ be $n$-dimensional. By expanding in a power series in $(1/2)\overrightarrow {r}^{\mathrm T}[K]\overrightarrow {r}$, show that

By expanding $e^{r_j Z_j}$ in a power series in $r_j Z_j$ for each $j$, show that

Let $D = \left\{ i_1, i_2, \ldots , i_{2m}\right\}$ be a set of $2m$ distinct integers each between 1 and $n$ and let $j_1, \ldots , j_{2m}$ be a permutation of $D$. Consider the term $r_{j_1} r_{j_2} \cdots r_{j_{2m}} \mathbb {E}\left[Z_{j_1} Z_{j_2} \cdots Z_{j_{2m}}\right]$ in (b) and let $(j_1, \ldots j_n)$ be a permutation of $D$. By comparing with the set of terms in (a) containing the same product $r_{j_1} r_{j_2} \cdots r_{j_{2m}}$, show that

where the sum is over all permutations $(j_1, j_2, \ldots , j_{2m})$ of the set $D$.

Find the number of permutations of $D$ that contain the same set of unordered pairs $(\left\{ j_1, j_2\right\} , \ldots , \left\{ j_{2m-1}, j_{2m}\right\} )$. For example, $(\left\{ 1,2\right\} , \left\{ 3,4\right\} )$ is the same set of unordered pairs as $(\left\{ 3,4\right\} , \left\{ 2,1\right\} )$. Show that

where the sum is over distinct sets of unordered pairs of the set $D$. Note: another way to say the same thing is that the sum is over the set of all permutations of $D$ for which $j_{2k-1} < j_{2k}$ for $1 \leq k \leq m$ and $j_{2k-1} < j_{2k+1}$ for $1 \leq k \leq m-1$.

To find $\mathbb {E}\left[Z_1^{j_1} \cdots Z_n^{j_n}\right]$, where $j_1+j_2+\cdots +j_n = 2m$, construct the rv s $U_1, \ldots , U_{2m}$, where $U_1, \ldots , U_{j_1}$ are all identically equal to $Z_1$, where $U_{j_1+1}, \ldots , U_{j_1+j_2}$ are identically equal to $Z_2$, etc., and use (3.114) to find $\mathbb {E}\left[U_1 U_2 \cdots U_{2m}\right]$. Use this formula to find $\mathbb {E}\left[Z_1^2 Z_2 Z_3\right]$, $\mathbb {E}\left[Z_1^2 Z_2^2\right]$, and $\mathbb {E}\left[Z_1\right]^4$.

Let $[K] = \left[\begin{smallmatrix} 0.75 & 0.25 \\ 0.25 & 0.75 \end{smallmatrix}\right]$. Show that 1 and $1/2$ are eigenvalues of $[K]$ and find the normalized eigenvectors. Express $[K]$ as $[Q\Lambda Q^{-1}]$, where $[\Lambda ]$ is diagonal and $[Q]$ is orthonormal.

Let $[K'] = \alpha [K]$ for real $\alpha \neq 0$. Find the eigenvalues and eigenvectors of $[K']$. Do not use brute force -- think!

Find the eigenvalues and eigenvectors of $[K^m]$, where $[K^m]$ is the $m$th power of $[K]$.

Let $X$ and $Y$ be zero-mean jointly Gaussian with variances $\sigma_X^2, \sigma_Y^2$, and normalized covariance $\rho$. Let $V = Y^3$. Find the conditional density $f_{X|V}(x \mid v)$.

Let $U = Y^2$ and find the conditional density of $f_{X|U}(x \mid u)$.

Let $(\overrightarrow {X}^{\mathrm T}, \overrightarrow {Y}^{\mathrm T})$ have a non-singular covariance matrix $[K]$. Show that $[K_X]$ and $[K_Y]$ are positive definite, and thus non-singular.

Let $X = (X_1,\ldots ,X_n)^{\mathrm T}$ and $Y = (Y_1,\ldots ,Y_m)^{\mathrm T}$ be zero-mean jointly-Gaussian rv s, with covariance matrices $[K_X]$ and $[K_Y]$ respectively, and let $[K]$ be the covariance matrix of the $(n+m)$-rv $(X_1,\ldots ,X_n,Y_1,\ldots ,Y_m)^{\mathrm T}$, partitioned as [K] = \left[\begin{smallmatrix} [K_X] & [K_{X \cdot Y}] \\[K_{X \cdot Y}]^{\mathrm T} & [K_Y] \end{smallmatrix}\right]. Assuming $[K]$ is non-singular, its inverse can be denoted in block form as

Show that the matrices $[B]$ and $[D]$ in (3.39) are also positive definite and thus non-singular.

Let $\overrightarrow {W}$ be a normalized IID Gaussian $n$-rv and let $\overrightarrow {Y}$ be a Gaussian $m$-rv. Suppose we would like the joint covariance $\mathbb {E}\left[\overrightarrow {W}\overrightarrow {Y}^{\mathrm T}\right]$ to be some arbitrary real-valued $n \times m$ matrix $[K]$. Find the matrix $[A]$ such that $\overrightarrow {Y} = [A]\overrightarrow {W}$ achieves the desired joint covariance. Note: this shows that any real-valued $n \times m$ matrix is the joint covariance matrix for some choice of random rv s.

Let $\overrightarrow {Z}$ be a zero-mean Gaussian $n$-rv with non-singular covariance $[K_Z]$, and let $\overrightarrow {Y}$ be a Gaussian $m$-rv. Suppose we would like the joint covariance $\mathbb {E}\left[\overrightarrow {Z}\overrightarrow {Y}^{\mathrm T}\right]$ to be some arbitrary $n \times m$ matrix $[K']$. Find the matrix $[B]$ such that $\overrightarrow {Y} = [B]\overrightarrow {Z}$ achieves the desired joint covariance. Note: this shows that any real-valued $n \times m$ matrix is the joint covariance matrix for some choice of rv s $\overrightarrow {Z}$ and $\overrightarrow {Y}$ where $[K_Z]$ is given (and non-singular).

Now assume that $\overrightarrow {Z}$ has a singular covariance matrix in (b). Explain the constraints this places on possible choices for the joint covariance $\mathbb {E}\left[\overrightarrow {Z}\overrightarrow {Y}^{\mathrm T}\right]$.

Let $\overrightarrow {W} = (W_1,W_2,\ldots ,W_{2n})^{\mathrm T}$ be a $2n$-dimensional IID normalized Gaussian rv. Let $S_{2n} = W_1^2+W_2^2+\cdots +W_{2n}^2$. Show that $S_{2n}$ is an $n$th-order Erlang rv with parameter $1/2$, i.e., that $f_{S_{2n}}(s) = 2^{-n}s^{n-1}e^{-s/2}/(n-1)!$.

Let $R_{2n} = \sqrt{S_{2n}}$. Find the probability density of $R_{2n}$.

Let $v_{2n}(r)$ be the volume of a $2n$-dimensional sphere of radius $r$ and let $b_{2n}(r)$ be the surface area of that sphere, i.e., $b_{2n}(r) = dv_{2n}(r)/dr$. The point of this exercise is to show how to calculate these quantities. By considering an infinitesimally thin spherical shell of thickness $\delta$ at radius $r$, show that

Calculate $b_{2n}(r)$ and $v_{2n}(r)$. Note that for any fixed $\delta \ll r$, the limiting ratio as $n \rightarrow \infty$ of that of a sphere of radius $r$ to that of a spherical shell of width $\delta$ and outer radius $r$ is 1.

Express $[B]$, $[C]$, and $[D]$, as defined in (3.39), in terms of $[K_X]$, $[K_Y]$ and $[K_{X \cdot Y}]$ by multiplying the block expression for $[K]$ by that for $[K]^{-1}$. You can check your solutions against those in (3.46)--(3.48). Hint: You can solve for $[B]$ and $[C]$ by looking at only two of the four block equations in $[KK^{-1}]$. You can use the symmetry between $X$ and $Y$ to solve for $[D]$.

Use your result in (a) for $[C]$ plus the symmetry between $X$ and $Y$ to show that

Theorem 3.5.2 shows that a zero-mean jointly-Gaussian, jointly-non-singular pair $X, Y$ can equivalently be represented as $X = [G]Y+V$, where $Y$ and $V$ are independent, with

or, reversing the roles of $X$ and $Y$, as $Y = [H]X+Z$, where $X$ and $Z$ are independent, with

For the formulations $X = [G]Y+V$ and $Y = [H]X+Z$, where $X$ and $Y$ are zero-mean, jointly Gaussian and have a non-singular combined covariance matrix, show that

Explain why $X_1,\ldots ,X_n,Z_1,\ldots ,Z_m$ must be jointly Gaussian rv s. Then explain why $X_1,\ldots ,X_n,Y_1,\ldots ,Y_m$ must be jointly Gaussian.

Show that if $[K_X]$ and $[K_Z]$ are non-singular, then the combined covariance matrix $[K]$ for $(X_1,\ldots ,X_n,Y_1,\ldots ,Y_m)^{\mathrm T}$ must be non-singular.

Verify (3.56) for $k=0$ by the use of induction on (3.54).

Verify (3.56) for $k=0$ directly from (3.55).

Verify (3.56) for $k>0$ by using induction on $k$.

Verify that

Let $X(t) = R\cos (2\pi f t+\theta )$ where $R$ is a Rayleigh rv and the rv $\theta$ is independent of $R$ and uniformly distributed over the interval 0 to $2\pi$. Show that $\mathbb {E}\left[X(t)\right] = 0$.

Show that $\mathbb {E}\left[X(t)X(t+\tau )\right] = \frac{1}{2}\mathbb {E}\left[R^2\right]\cos (2\pi f \tau )$.

Show that $\left\{ X(t); t \in \mathbb {R}\right\}$ is a Gaussian process.

Let $\overrightarrow {Z} = (Z_1,\ldots ,Z_n)^{\mathrm T}$ be a circularly-symmetric $n$-rv. Show that $Z_k$ is circularly symmetric for each $k$, $1 \leq k \leq n$.

Show that $Z_1+Z_2$ is circularly symmetric. For any complex $n$-vector $\overrightarrow {c}$, show that $\overrightarrow {c}^{\mathrm T}\overrightarrow {Z}$ is a circularly symmetric rv.

Show that $\mathbb {E}\left[AA^*\right] = \sigma_1^2+\sigma_2^2$.

Show that

Show that $\mathbb {E}\left[(AA^*)^2\right] \geq 2\left(\mathbb {E}\left[AA^*\right]\right)^2$ with equality if and only if $A_1$ and $A_2$ are IID.

Show that $\operatorname {Var}\left[AA^*\right] \geq \left(\mathbb {E}\left[AA^*\right]\right)^2$.