Linear, Bilinear, and Quadratic Forms

Show that a symmetric bilinear form $x'Ay$ (in $n$-dimensional vectors $x$ and $y$) can be expressed in terms of the corresponding quadratic form, that is, the quadratic form whose matrix is $A$. Do so by verifying that

Show that corresponding to any quadratic form $x'Ax$ (in the $n$-dimensional vector $x$) there exists a unique upper triangular matrix $B$ such that $x'Ax$ and $x'Bx$ are identically equal, and express the elements of $B$ in terms of the elements of $A$.

Show, by example, that the sum of two positive semidefinite matrices can be positive definite.

Show, via an example, that there exist (nonsymmetric) nonsingular positive semidefinite matrices.

Show, by example, that there exist an $n \times n$ positive semidefinite matrix $A$ and an $n \times m$ matrix $P$ (where $m < n$) such that $P'AP$ is positive definite.

For an $n \times n$ matrix $A$ and an $n \times m$ matrix $P$, it is the case that (1) if $A$ is nonnegative definite, then $P'AP$ is nonnegative definite; (2) if $A$ is nonnegative definite and $\operatorname {rk}\left(P\right) < m$, then $P'AP$ is positive semidefinite; and (3) if $A$ is positive definite and $\operatorname {rk}\left(P\right) = m$, then $P'AP$ is positive definite. Convert these results, which are for nonnegative definite (positive definite or positive semidefinite) matrices, into equivalent results for nonpositive definite matrices.

Let $\{ X_1, \ldots , X_r\}$ represent a set of matrices from a linear space $\mathcal{V}$. And, let $A = \{ a_{ij}\}$ represent the $r \times r$ matrix whose $ij$th element is $\left\langle X_i, X_j \right\rangle$ -- this matrix is referred to as the Gram matrix (or the Gramian) of the set $\{ X_1, \ldots , X_r\}$ and its determinant is referred to as the Gramian (or the Gram determinant) of $\{ X_1, \ldots , X_r\}$.

Let $A = \{ a_{ij}\}$ represent an $n \times n$ symmetric positive definite matrix, and let $B = \{ b_{ij}\} = A^{-1}$. Show that, for $i = 1, \ldots , n$,

with equality holding if and only if, for all $j \neq i$, $a_{ij} = 0$.

Let $A$ represent an $m \times n$ matrix and $D$ a diagonal matrix such that $A = PDQ$ for some matrix $P$ of full column rank and some matrix $Q$ of full row rank. Show that $\operatorname {rk}\left(A\right)$ equals the number of nonzero diagonal elements in $D$.

Let $A$ represent an $n \times n$ symmetric idempotent matrix and $V$ an $n \times n$ symmetric positive definite matrix. Show that $\operatorname {rk}\left(AVA\right) = \operatorname {tr}(A)$.

Show that if an $n \times n$ matrix $A$ is such that $x'Ax \neq 0$ for every $n \times 1$ nonnull vector $x$, then $A$ is either positive definite or negative definite.

Let $A$ represent an $n \times n$ symmetric nonnegative definite matrix of rank $r$. Then, there exists an $n \times r$ matrix $B$ (of rank $r$) such that $A = BB'$. Let $X$ represent any $n \times m$ matrix (where $m \geq r$) such that $A = XX'$.

Show that if a symmetric matrix $A$ has a nonnegative definite generalized inverse, then $A$ is nonnegative definite.

Suppose that an $n \times n$ matrix $A$ has an LDU decomposition, say $A = LDU$, and let $d_1, d_2, \ldots , d_n$ represent the diagonal elements of the diagonal matrix $D$. Show that

Suppose that an $n \times n$ (symmetric) matrix $A$ has a unique $U'DU$ decomposition, say $A = U'DU$. Use the result of Part (b) of Exercise 14.16 to show that $A$ has no LDU decompositions other than $A = U'DU$.

Show that if a nonsingular matrix has an LDU decomposition, then that decomposition is unique.

Let $A$ represent an $n \times n$ matrix (where $n \geq 2$). By for instance using the results of Exercises 14.16, 14.17, and 14.18, show that if $A$ has a unique LDU decomposition or (in the special case where $A$ is symmetric) a unique $U'DU$ decomposition, then the leading principal submatrices (of $A$) of orders $1, 2, \ldots , n-1$ are nonsingular and have unique LDU decompositions.

Show, by example, that there exist $n \times n$ (nonsymmetric) positive semidefinite matrices that do not have LDU decompositions.

Let $A$ represent an $n \times n$ nonnegative definite (possibly nonsymmetric) matrix that has an LDU decomposition, say $A = LDU$. Show that the diagonal elements of the diagonal matrix $D$ are nonnegative.

Let $A$ represent an $m \times k$ matrix of full column rank. And, let $A = QR$ represent the QR decomposition of $A$; that is, let $Q$ represent the unique $m \times k$ matrix whose columns are orthonormal with respect to the usual inner product and let $R$ represent the unique $k \times k$ upper triangular matrix with positive diagonal elements such that $A = QR$. Show that $A'A = R'R$ (so that $A'A = R'R$ is the Cholesky decomposition of $A'A$).

Let $A$ represent an $m \times k$ matrix of rank $r$ (where $r$ is possibly less than $k$). Consider the decomposition $A = QR_1$, where $Q$ is an $m \times r$ matrix with orthonormal columns and $R_1$ is an $r \times k$ submatrix whose rows are the $r$ nonnull rows of a $k \times k$ upper triangular matrix $R$ having $r$ positive diagonal elements and $n - r$ null rows. (Such a decomposition can be obtained by using the results of Exercise 6.4 -- refer to Exercise 6.5.) Generalize the result of Exercise 14.23 by showing that if the inner product with respect to which the columns of $Q$ are orthonormal is the usual inner product, then $A'A = R'R$ (so that $A'A = R'R$ is the Cholesky decomposition of $A'A$).

Let $A = \{ a_{ij}\}$ represent an $n \times n$ matrix that has an LDU decomposition, say $A = LDU$. And, define $G = U^{-1}D^{-}L^{-1}$ (which is a generalized inverse of $A$).

Verify that a principal submatrix of a skew-symmetric matrix is skew-symmetric.

Show, via an example, that (for $n > 1$) there exist $n \times n$ (nonsymmetric) positive definite matrices $A$ and $B$ such that $\operatorname {tr}(AB) < 0$.

Let $A = \{ a_{ij}\}$ and $B = \{ b_{ij}\}$ represent $n \times n$ matrices, and take $C$ to be the $n \times n$ matrix whose $ij$th element $c_{ij} = a_{ij}b_{ij}$ is the product of the $ij$th elements of $A$ and $B$ -- $C$ is the so-called Hadamard product of $A$ and $B$. Show that if $A$ is nonnegative definite and $B$ is symmetric nonnegative definite, then $C$ is nonnegative definite. Show further that if $A$ is positive definite and $B$ is symmetric positive definite, then $C$ is positive definite.

Let $A_1, A_2, \ldots , A_k$ and $B_1, B_2, \ldots , B_k$ represent $n \times n$ symmetric nonnegative definite matrices. Show that $\operatorname {tr}(\sum_{i=1}^{k} A_iB_i) \geq 0$, with equality holding if and only if, for $i = 1, 2, \ldots , k$, $A_iB_i = 0$, thereby generalizing the results of Part (b) of Exercise 14.28.

Let $A$ represent a symmetric nonnegative definite matrix that has been partitioned as

where $T$ (and hence $W$) is square. Show that $VT^{-}U$ and $UW^{-}V$ are symmetric and nonnegative definite.

Show, via an example, that there exists an $(m+n) \times (m+n)$ (nonsymmetric) positive semidefinite matrix $A$ of the form $A = \left(\begin{smallmatrix} T & U \\ V & W \end{smallmatrix}\right)$, where $T$ is of dimensions $m \times m$, $W$ of dimensions $n \times n$, $U$ of dimensions $m \times n$, and $V$ of dimensions $n \times m$, for which $C(U) \not\subset C(T)$ and/or $R(V) \not\subset R(T)$, the expression $\operatorname {rk}\left(T\right) + \operatorname {rk}\left(W-VT^{-}U\right)$ does not necessarily equal $\operatorname {rk}\left(A\right)$, and the formula

where $Q = W - VT^{-}U$, does not necessarily give a generalized inverse of $A$.

Show, via an example, that there exists an $(m+n) \times (m+n)$ symmetric partitioned matrix $A$ of the form $A = \left(\begin{smallmatrix} T & U \\ U' & W \end{smallmatrix}\right)$, where $T$ is of dimensions $m \times m$, $U$ of dimensions $m \times n$, and $W$ of dimensions $n \times n$, such that $T$ is nonnegative definite and (depending on the choice of $T^{-}$) the Schur complement $W - U'T^{-}U$ of $T$ relative to $T^{-}$ is nonnegative definite, but $A$ is not nonnegative definite.

An $n \times n$ matrix $A = \{ a_{ij}\}$ is said to be diagonally dominant if, for $i = 1, 2, \ldots , n$, $\left|a_{ii}\right| > \sum_{j=1(j \neq i)}^{n} \left|a_{ij}\right|$. (In the degenerate special case where $n = 1$, $A$ is said to be diagonally dominant if it is nonnull).

Let $A = \{ a_{ij}\}$ represent an $n \times n$ symmetric positive definite matrix. Show that $\det (A) \leq \prod_{i=1}^{n} a_{ii}$, with equality holding if and only if $A$ is diagonal.

Let $A = \left(\begin{smallmatrix} a & b \\ c & d \end{smallmatrix}\right)$, where $a, b, c$, and $d$ are scalars.

By, for example, making use of the result of Exercise 14.38, show that if an $n \times n$ matrix $A = \{ a_{ij}\}$ is symmetric positive definite, then, for $j \neq i = 1, \ldots , n$,

Show, by example, that it is possible for the determinants of both leading principal submatrices of a $2 \times 2$ symmetric matrix to be nonnegative without the matrix being nonnegative definite and that, for $n \geq 3$, it is possible for the determinants of all $n$ leading principal submatrices of an $n \times n$ symmetric matrix to be nonnegative and for the matrix to be nonsingular without the matrix being nonnegative definite.

Let $\mathcal{V}$ represent a subspace of $\mathbb {R}^{n \times 1}$ of dimension $r$ (where $r \geq 1$). Take $B = (b_1, b_2, \ldots , b_r)$ to be any $n \times r$ matrix whose columns $b_1, b_2, \ldots , b_r$ form a basis for $\mathcal{V}$, and let $L$ represent any left inverse of $B$. Let $g$ represent a function that assigns the value $x * y$ to an arbitrary pair of vectors $x$ and $y$ in $\mathcal{V}$.

Let $\mathcal{V}$ represent a linear space of $m \times n$ matrices, and let $A \cdot B$ represent the value assigned by a quasi-inner product to any pair of matrices $A$ and $B$ (in $\mathcal{V}$). Show that the set

which comprises every matrix in $\mathcal{V}$ with a zero quasi norm, is a linear space.

Let $W$ represent an $m \times m$ symmetric positive definite matrix and $V$ an $n \times n$ symmetric positive definite matrix.

Let $A$ represent a $q \times p$ matrix, $B$ a $p \times n$ matrix, and $C$ an $m \times q$ matrix. Show that

Let $\mathcal{U}$ represent a subspace of $\mathbb {R}^{n \times 1}$, let $X$ represent an $n \times p$ matrix whose columns span $\mathcal{U}$, and let $W$ and $V$ represent $n \times n$ symmetric positive definite matrices. Show that each of the following two conditions is necessary and sufficient for the projection $P_{X,W}y$ of $y$ on $\mathcal{U}$ with respect to $W$ to be the same (for every $y$ in $\mathbb {R}^{n}$) as the projection $P_{X,V}y$ of $y$ on $\mathcal{U}$ with respect to $V$:

Let $y$ represent an $n$-dimensional column vector, let $\mathcal{U}$ represent a subspace of $\mathbb {R}^{n \times 1}$, and let $X$ represent an $n \times p$ matrix whose columns span $\mathcal{U}$. ``Recall'' that, for any $m \times n$ matrix $L$, for any subspace $\mathcal{W}$ of $\mathbb {R}^{n \times 1}$, and for $\mathcal{V} = \{ v \in \mathbb {R}^{m} : v = Lx \text{ for some } x \in \mathcal{W}\}$,

where $H = L'L$ (and where $x$ represents an arbitrary $n \times 1$ vector), and

where $Z$ is any $n \times q$ matrix whose columns span $\mathcal{W}$. Use results (*) and (**) to show that, for any $n \times n$ symmetric nonnegative definite matrix $W$, $y \perp_W \mathcal{U}$ if and only if $X'Wy = 0$.

Let $W$ represent an $n \times n$ symmetric nonnegative definite matrix.

Let $\mathcal{U}$ represent a subspace of $\mathbb {R}^{n \times 1}$, let $A$ represent an $n \times n$ matrix and $W$ an $n \times n$ symmetric nonnegative definite matrix, and let $X$ represent any $n \times p$ matrix whose columns span $\mathcal{U}$.

Let $A$ represent an $n \times n$ matrix and $W$ an $n \times n$ symmetric nonnegative definite matrix.