Eigenvalues and Eigenvectors

Show that an $n \times n$ skew-symmetric matrix $\mathbf{A}$ has no nonzero eigenvalues.

Let $\mathbf{A}$ represent an $n \times n$ matrix, $\mathbf{B}$ a $k \times k$ matrix, and $\mathbf{X}$ an $n \times k$ matrix such that $\mathbf{A}\mathbf{X} = \mathbf{X}\mathbf{B}$.

Let $p(\lambda )$ represent the characteristic polynomial of an $n \times n$ matrix $\mathbf{A}$, and let $c_{0}, c_{1}, c_{2}, \ldots , c_{n}$ represent the respective coefficients of the characteristic polynomial, so that

(for $\lambda \in \mathbb {R}$). Further, let $\mathbf{P}$ represent the $n \times n$ matrix obtained from $p(\lambda )$ by formally replacing the scalar $\lambda$ with the $n \times n$ matrix $\mathbf{A}$ (and by setting $\mathbf{A}^{0} = \mathbf{I}_{n}$). That is, let

Show that $\mathbf{P} = \mathbf{0}$ (a result that is known as the Cayley-Hamilton theorem) by carrying out the following four steps.

Let $c_{0}, c_{1}, \ldots , c_{n-1}, c_{n}$ represent the respective coefficients of the characteristic polynomial $p(\lambda )$ of an $n \times n$ matrix $\mathbf{A}$ [so that $p(\lambda ) = c_{0} + c_{1}\lambda + \cdots + c_{n-1}\lambda^{n-1} + c_{n}\lambda^{n}$ (for $\lambda \in \mathbb {R}$)]. Using the result of Exercise 3 (the Cayley-Hamilton theorem), show that if $\mathbf{A}$ is nonsingular, then $c_{0} \neq 0$, and

Show that if an $n \times n$ matrix $\mathbf{B}$ is similar to an $n \times n$ matrix $\mathbf{A}$, then (1) $\mathbf{B}^{k}$ is similar to $\mathbf{A}^{k}$ ($k = 2, 3, \ldots$) and (2) $\mathbf{B}'$ is similar to $\mathbf{A}'$.

Show that if an $n \times n$ matrix $\mathbf{B}$ is similar to an ($n \times n$) idempotent matrix, then $\mathbf{B}$ is idempotent.

Let $\mathbf{A} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$ and $\mathbf{B} = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$. Show that $\mathbf{B}$ has the same rank, determinant, trace, and characteristic polynomial as $\mathbf{A}$, but that, nevertheless, $\mathbf{B}$ is not similar to $\mathbf{A}$.

Expand on the result of Exercise 7 by showing (for an arbitrary positive integer $n$) that for an $n \times n$ matrix $\mathbf{B}$ to be similar to an $n \times n$ matrix $\mathbf{A}$ it is not sufficient for $\mathbf{B}$ to have the same rank, determinant, trace, and characteristic polynomial as $\mathbf{A}$.

Let $\mathbf{A}$ represent an $n \times n$ matrix, $\mathbf{B}$ a $k \times k$ matrix, and $\mathbf{X}$ an $n \times k$ matrix such that $\mathbf{A}\mathbf{X} = \mathbf{X}\mathbf{B}$. Show that if $\mathbf{X}$ is of full column rank, then there exists an orthogonal matrix $\mathbf{Q}$ such that $\mathbf{Q}'\mathbf{A}\mathbf{Q} = \begin{pmatrix} \mathbf{T}_{11} & \mathbf{T}_{12} \\ \mathbf{0} & \mathbf{T}_{22} \end{pmatrix}$, where $\mathbf{T}_{11}$ is a $k \times k$ matrix that is similar to $\mathbf{B}$.

Show that if $0$ is an eigenvalue of an $n \times n$ matrix $\mathbf{A}$, then its algebraic multiplicity is greater than or equal to $n - \operatorname {rk}\left(\mathbf{A}\right)$.

Let $\mathbf{A}$ represent an $n \times n$ matrix. Show that if a scalar $\lambda$ is an eigenvalue of $\mathbf{A}$ of algebraic multiplicity $\gamma$, then $\operatorname {rk}\left(\mathbf{A} - \lambda \mathbf{I}\right) \geq n - \gamma$.

Let $\gamma_{1}$ represent the algebraic multiplicity and $\nu_{1}$ the geometric multiplicity of $0$ when $0$ is regarded as an eigenvalue of an $n \times n$ (singular) matrix $\mathbf{A}$. And let $\gamma_{2}$ represent the algebraic multiplicity and $\nu_{2}$ the geometric multiplicity of $0$ when $0$ is regarded as an eigenvalue of $\mathbf{A}^{2}$. Show that if $\nu_{1} = \gamma_{1}$, then $\nu_{2} = \gamma_{2} = \nu_{1}$.

Let $\mathbf{x}_{1}$ and $\mathbf{x}_{2}$ represent eigenvectors of an $n \times n$ matrix $\mathbf{A}$, and let $c_{1}$ and $c_{2}$ represent nonzero scalars. Under what circumstances is the vector $\mathbf{x} = c_{1}\mathbf{x}_{1} + c_{2}\mathbf{x}_{2}$ an eigenvector of $\mathbf{A}$?

Let $\mathbf{A}$ represent an $n \times n$ matrix, and suppose that there exists an $n \times n$ nonsingular matrix $\mathbf{Q}$ such that $\mathbf{Q}^{-1}\mathbf{A}\mathbf{Q} = \mathbf{D}$ for some diagonal matrix $\mathbf{D} = \{ d_{i}\}$. Further, for $i = 1, \ldots , n$, let $\mathbf{r}_{i}'$ represent the $i$th row of $\mathbf{Q}^{-1}$. Show

Show that if an $n \times n$ nonsingular matrix $\mathbf{A}$ is diagonalized by an $n \times n$ nonsingular matrix $\mathbf{Q}$, then $\mathbf{A}^{-1}$ is also diagonalized by $\mathbf{Q}$.

Let $\mathbf{A}$ represent an $n \times n$ matrix whose spectrum comprises $k$ eigenvalues $\lambda_{1}, \ldots , \lambda_{k}$ with algebraic multiplicities $\gamma_{1}, \ldots , \gamma_{k}$, respectively, that sum to $n$. Show that $\mathbf{A}$ is diagonalizable if and only if, for $i = 1, \ldots , k$, $\operatorname {rk}\left(\mathbf{A} - \lambda_{i}\mathbf{I}\right) = n - \gamma_{i}$.

Let $\mathbf{A}$ represent an $n \times n$ symmetric matrix with not-necessarily-distinct eigenvalues $d_{1}, \ldots , d_{n}$ that have been ordered so that $d_{1} \leq d_{2} \leq \cdots \leq d_{n}$. And, let $\mathbf{Q}$ represent an $n \times n$ orthogonal matrix such that $\mathbf{Q}'\mathbf{A}\mathbf{Q} = \diag (d_{1}, \ldots , d_{n})$ --- the existence of which is guaranteed. Further, for $m = 2, \ldots , n-1$, define $S_{m} = \{ \mathbf{x} \in \mathbb {R}^{n \times 1} : \mathbf{x} \neq \mathbf{0}, \mathbf{Q}_{m}'\mathbf{x} = \mathbf{0}\}$ and $T_{m} = \{ \mathbf{x} \in \mathbb {R}^{n \times 1} : \mathbf{x} \neq \mathbf{0}, \mathbf{P}_{m}'\mathbf{x} = \mathbf{0}\}$, where $\mathbf{Q}_{m} = (\mathbf{q}_{1}, \ldots , \mathbf{q}_{m-1})$ and $\mathbf{P}_{m} = (\mathbf{q}_{m+1}, \ldots , \mathbf{q}_{n})$. Show that, for $m = 2, \ldots , n-1$,

Let $\mathbf{A}$ represent an $n \times n$ symmetric matrix, and adopt the following notation: $d_{1}, \ldots , d_{n}$ are the not-necessarily-distinct eigenvalues of $\mathbf{A}$, $\mathbf{q}_{1}, \ldots , \mathbf{q}_{n}$ are orthonormal eigenvectors corresponding to $d_{1}, \ldots , d_{n}$, respectively, $\mathbf{Q} = (\mathbf{q}_{1}, \ldots , \mathbf{q}_{n})$, $\{ \lambda_{1}, \ldots , \lambda_{k}\}$ is the spectrum of $\mathbf{A}$; and, for $j = 1, \ldots , k$, $S_{j} = \{ i : d_{i} = \lambda_{j}\}$, $\mathbf{E}_{j} = \sum_{i \in S_{j}} \mathbf{q}_{i}\mathbf{q}_{i}'$, and $\mathbf{Q}_{j} = (\mathbf{q}_{i_{1}}, \ldots , \mathbf{q}_{i_{\nu_{j}}})$, where $i_{1}, \ldots , i_{\nu_{j}}$ denote the elements of $S_{j}$.

Let $\mathbf{A}$ represent an $n \times n$ symmetric matrix, and let $d_{1}, \ldots , d_{n}$ represent the (not-necessarily-distinct) eigenvalues of $\mathbf{A}$. Show that $\lim_{k \rightarrow \infty } \mathbf{A}^{k} = \mathbf{0}$ if and only if, for $i = 1, \ldots , n$, $\left|d_{i}\right| < 1$.

Show that, for any positive integer $n$ that is divisible by 2, there exists an $n \times n$ orthogonal matrix that has no eigenvalues.

Let $\mathbf{Q}$ represent an $n \times n$ orthogonal matrix, and let $p(\lambda )$ represent the characteristic polynomial of $\mathbf{Q}$. Show that (for $\lambda \neq 0$)

Let $\mathbf{A}$ represent an $n \times n$ matrix, and suppose that the scalar 1 is an eigenvalue of $\mathbf{A}$ of geometric multiplicity $\nu$. Show that $\nu \leq \operatorname {rk}\left(\mathbf{A}\right)$ and that if $\nu = \operatorname {rk}\left(\mathbf{A}\right)$, then $\mathbf{A}$ is idempotent.

Let $\mathbf{A}$ represent an $n \times n$ nonsingular matrix. And, let $\lambda$ represent an eigenvalue of $\mathbf{A}$, and $\mathbf{x}$ represent an eigenvector of $\mathbf{A}$ corresponding to $\lambda$. Show that $\left|\mathbf{A}\right|/\lambda$ is an eigenvalue of $\operatorname {adj}(\mathbf{A})$ and that $\mathbf{x}$ is an eigenvector of $\operatorname {adj}(\mathbf{A})$ corresponding to $\left|\mathbf{A}\right|/\lambda$.

Let $\mathbf{A}$ represent an $n \times n$ matrix, and let $p(\lambda )$ represent the characteristic polynomial of $\mathbf{A}$. And, let $\lambda_{1}, \ldots , \lambda_{k}$ represent the distinct eigenvalues of $\mathbf{A}$, and $\gamma_{1}, \ldots , \gamma_{k}$ represent their respective algebraic multiplicities, so that (for all $\lambda$)

for some polynomial $q(\lambda )$ (of degree $n - \sum_{j=1}^{k} \gamma_{j}$) that has no real roots. Further, define $\mathbf{B} = \mathbf{A} - \lambda_{1}\mathbf{U}\mathbf{V}'$, where $\mathbf{U} = (\mathbf{u}_{1}, \ldots , \mathbf{u}_{\gamma_{1}})$ is an $n \times \gamma_{1}$ matrix whose columns $\mathbf{u}_{1}, \ldots , \mathbf{u}_{\gamma_{1}}$ are (not necessarily linearly independent) eigenvectors of $\mathbf{A}$ corresponding to $\lambda_{1}$ and where $\mathbf{V} = (\mathbf{v}_{1}, \ldots , \mathbf{v}_{\gamma_{1}})$ is an $n \times \gamma_{1}$ matrix such that $\mathbf{V}'\mathbf{U}$ is diagonal.

Let $\mathbf{A}$ represent an $m \times n$ matrix of rank $r$. And, take $\mathbf{P}$ to be any $m \times m$ orthogonal matrix and $\mathbf{D}_{1}$ to be any $r \times r$ nonsingular diagonal matrix such that

Further, partition $\mathbf{P}$ as $\mathbf{P} = (\mathbf{P}_{1}, \mathbf{P}_{2})$, where $\mathbf{P}_{1}$ has $r$ columns, and let $\mathbf{Q} = (\mathbf{Q}_{1}, \mathbf{Q}_{2})$, where $\mathbf{Q}_{1} = \mathbf{A}'\mathbf{P}_{1}\mathbf{D}_{1}^{-1}$ and where $\mathbf{Q}_{2}$ is any $n \times (n-r)$ matrix such that $\mathbf{Q}_{1}'\mathbf{Q}_{2} = \mathbf{0}$. Show that

Let $\mathbf{A}$ represent an $m \times n$ matrix. And, let $\mathbf{P}$ represent an $m \times m$ orthogonal matrix, $\mathbf{Q}$ an $n \times n$ orthogonal matrix, and $\mathbf{D}_{1} = \{ s_{i}\}$ an $r \times r$ diagonal matrix with (strictly) positive diagonal elements such that $\mathbf{P}'\mathbf{A}\mathbf{Q} = \begin{pmatrix} \mathbf{D}_{1} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{pmatrix}$; denote by $\mathbf{p}_{1}, \ldots , \mathbf{p}_{r}$ the first through $r$th columns of $\mathbf{P}$ and by $\mathbf{q}_{1}, \ldots , \mathbf{q}_{r}$ the first through $r$th columns of $\mathbf{Q}$; and define $\alpha_{1}, \ldots , \alpha_{k}$ to be the distinct values represented among $s_{1}, \ldots , s_{r}$. Then, the singular value decomposition of $\mathbf{A}$ is $\mathbf{A} = \sum_{j=1}^{k} \alpha_{j}\mathbf{U}_{j}$, where (for $j = 1, \ldots , k$) $\mathbf{U}_{j} = \sum_{i \in L_{j}} \mathbf{p}_{i}\mathbf{q}_{i}'$ with $L_{j} = \{ i : s_{i} = \alpha_{j}\}$. Show that the matrices $\mathbf{U}_{1}, \ldots , \mathbf{U}_{k}$, which appear in the singular value decomposition, are such that $\mathbf{U}_{j}\mathbf{U}_{j}'\mathbf{U}_{j} = \mathbf{U}_{j}$ (for $j = 1, \ldots , k$) and $\mathbf{U}_{t}'\mathbf{U}_{j} = \mathbf{0}$ and $\mathbf{U}_{t}\mathbf{U}_{j}' = \mathbf{0}$ (for $t \neq j = 1, \ldots , k$).

Let $\mathbf{A}$ represent an $m \times n$ matrix. And, as in Exercise 27, take $\mathbf{P}$ to be an $m \times m$ orthogonal matrix, $\mathbf{Q}$ an $n \times n$ orthogonal matrix, and $\mathbf{D}_{1}$ an $r \times r$ nonsingular diagonal matrix such that $\mathbf{P}'\mathbf{A}\mathbf{Q} = \begin{pmatrix} \mathbf{D}_{1} & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{pmatrix}$. Further, partition $\mathbf{P}$ and $\mathbf{Q}$ as $\mathbf{P} = (\mathbf{P}_{1}, \mathbf{P}_{2})$ and $\mathbf{Q} = (\mathbf{Q}_{1}, \mathbf{Q}_{2})$, where each of the matrices $\mathbf{P}_{1}$ and $\mathbf{Q}_{1}$ has $r$ columns. Show that $C(\mathbf{A}) = C(\mathbf{P}_{1})$ and that $N(\mathbf{A}) = C(\mathbf{Q}_{2})$.

Let $\mathbf{A}_{1}, \ldots , \mathbf{A}_{k}$ represent $n \times n$ not-necessarily-symmetric matrices, each of which is diagonalizable. Show that if $\mathbf{A}_{1}, \ldots , \mathbf{A}_{k}$ commute in pairs, then $\mathbf{A}_{1}, \ldots , \mathbf{A}_{k}$ are simultaneously diagonalizable.

Let $\mathbf{V}$ represent an $n \times n$ symmetric nonnegative definite matrix, $\mathbf{X}$ an $n \times p$ matrix of rank $r$, and $\mathbf{d}$ a $p \times 1$ vector. Using the results of Exercise 19.11 (or otherwise), show that each of the following three conditions is necessary and sufficient for the vector $\mathbf{X}(\mathbf{X}'\mathbf{X})^{-}\mathbf{d}$ to be a solution, for every $\mathbf{d} \in C(\mathbf{X}')$, to the problem of minimizing the quadratic form $\mathbf{a}'\mathbf{V}\mathbf{a}$ (in $\mathbf{a}$) subject to $\mathbf{X}'\mathbf{a} = \mathbf{d}$:

Let $\mathbf{A}$ represent an $n \times n$ symmetric matrix, and let $\mathbf{B}$ represent an $n \times n$ symmetric positive definite matrix. And, let $\lambda_{\max }$ and $\lambda_{\min }$ represent, respectively, the largest and smallest roots of $\left|\mathbf{A} - \lambda \mathbf{B}\right|$. Show that

for every nonnull vector $\mathbf{x}$ in $\mathbb {R}^{n}$.

Let $\mathbf{A}$ represent an $n \times n$ symmetric matrix, and let $\mathbf{B}$ represent an $n \times n$ symmetric positive definite matrix. Show that $\mathbf{A} - \mathbf{B}$ is nonnegative definite if and only if all $n$ (not necessarily distinct) roots of $\left|\mathbf{A} - \lambda \mathbf{B}\right|$ are greater than or equal to 1 and is positive definite if and only if all $n$ roots are (strictly) greater than 1.