Singular Value Decomposition

Following the derivation in the text, find an SVD for $C = \begin{pmatrix} -4& -6 \\ 3& -8 \end{pmatrix}$.

If $\sigma_{1}\geq \sigma_{2}\geq \cdots \geq \sigma_{r}$ are the nonzero singular values of $A$, then it can be shown that the function $\nu_{k}(A) = \left(\sigma_{1}^{2}+\sigma_{2}^{2}+\cdots +\sigma_{k}^{2}\right)^{1/2}$ defines a unitarily invariant norm (Exercise 5.6.9) for $\mathbb {R}^{m\times n}$ (or $\mathbb {C}^{m\times n}$) for each $k=1,2,\ldots ,r$. Explain why the 2-norm and the Frobenius norm are the extreme cases in the sense that $\left\| A\right\|_{2}^{2} = \sigma_{1}^{2}$ and $\left\| A\right\|_{F}^{2} = \sigma_{1}^{2}+\sigma_{2}^{2}+\cdots +\sigma_{r}^{2}$.

Explain why $\left\| A\right\|_{2} \leq \left\| A\right\|_{F} \leq \sqrt{n}\left\| A\right\|_{2}$ (the $(2,F)$ and $(F,2)$ entries of the matrix-norm equivalence table from Exercise 5.1.8/5.12.3's own bound matrix).

Prove that if $\sigma_{1}\geq \sigma_{2}\geq \cdots \geq \sigma_{r}$ are the nonzero singular values of a rank $r$ matrix $A$, and if $\left\| E\right\|_{2} < \sigma_{r}$, then $\operatorname {rk}\left(A+E\right) \geq \operatorname {rk}\left(A\right)$.

Extend the result on the image of the unit sphere to include singular and rectangular matrices by showing that if $\sigma_{1}\geq \sigma_{2}\geq \cdots \geq \sigma_{r}>0$ are the nonzero singular values of $A_{m\times n}$, then the image $A(S_{2}) \subset \mathbb {R}^{m}$ of the unit 2-sphere $S_{2}\subset \mathbb {R}^{n}$ is an ellipsoid (possibly degenerate) in which the $k$th semiaxis is $\sigma_{k}U_{*k} = AV_{*k}$, where $U_{*k}$ and $V_{*k}$ are respective left-hand and right-hand singular vectors for $A$.

Prove that if $\sigma_{r}$ is the smallest nonzero singular value of $A_{m\times n}$, then

Extend the bound $\kappa^{-1}\left\| e\right\| /\left\| b\right\| \leq \left\| x-\tilde{x}\right\| /\left\| x\right\| \leq \kappa \left\| e\right\| /\left\| b\right\|$ (where $\kappa = \left\| A\right\| \left\| A^{-1}\right\|$, for a nonsingular system's uncertainty analysis) to include singular and rectangular matrices by showing that if $x$ and $\tilde{x}$ are the respective minimum 2-norm solutions of consistent systems $Ax=b$ and $A\tilde{x} = \tilde{b} = b-e$, then

Can the same reasoning as in the nonsingular uncertainty example be used to argue that for $\left\| \star \right\|_{2}$, the upper and lower bounds are attainable for every $A$?

Prove that if $\left|\epsilon \right| < \sigma_{r}^{2}$ for the smallest nonzero singular value of $A_{m\times n}$, then $(A^{T}A+\epsilon I)^{-1}$ exists, and $\lim_{\epsilon \to 0}(A^{T}A+\epsilon I)^{-1}A^{T} = A^{\dagger }$.

Consider a system $Ax=b$ in which $A = \begin{pmatrix} .835& .667 \\ .333& .266 \end{pmatrix}$, and suppose $b$ is subject to an uncertainty $e$. Using $\infty$-norms, determine the directions of $b$ and $e$ that give rise to the worst-case scenario $\left\| x-\tilde{x}\right\|_{\infty }/\left\| x\right\|_{\infty } = \kappa_{\infty }\left\| e\right\|_{\infty }/\left\| b\right\|_{\infty }$.

An ill-conditioned matrix is suspected when a small pivot $u_{ii}$ emerges during the LU factorization of $A$ (because $\left[U^{-1}\right]_{ii} = 1/u_{ii}$ is then large), but this is not an absolute test.

Bound the relative uncertainty in the solution of a nonsingular system $Ax=b$ for which there is some uncertainty in $A$ but not in $b$, by showing that if $(A-E)\tilde{x}=b$, where $\alpha = \left\| A^{-1}E\right\| < 1$ for any matrix norm with $\left\| I\right\| =1$, then

Now bound the relative uncertainty in the solution of a nonsingular system $Ax=b$ for which there is some uncertainty in both $A$ and $b$, by showing that if $(A-E)\tilde{x} = b-e$, where $\alpha = \left\| A^{-1}E\right\| < 1$ for any matrix norm with $\left\| I\right\| =1$, then

Consider the matrix $A = \begin{pmatrix} -4& -2& -4& -2 \\ 2& -2& 2& 1 \\ -4& 1& -4& -2 \end{pmatrix}$.

For matrix $A$ in Exercise 5.11.8, and for $b = \begin{pmatrix} -12& 3& -9 \end{pmatrix}^{T}$, find the solution of $Ax=b$ that has minimum euclidean norm.

Suppose $A = URV^{T}$ is a URV factorization (so it could be an SVD) of an $m\times n$ matrix of rank $r$, and suppose $U$ is partitioned as $U = \left[U_{1}\mid U_{2}\right]$, where $U_{1}$ is $m\times r$. Prove that $P = U_{1}U_{1}^{T} = AA^{\dagger }$ is the projector onto $\mathcal{R}(A)$ along $\mathcal{N}(A^{T})$.

Establish the following properties of $A^{\dagger }$.

Explain why $A^{\dagger } = A^{D}$ if and only if $A$ is an RPN matrix (i.e., $\mathcal{R}(A)\perp \mathcal{N}(A)$).

Let $X, Y \in \mathbb {R}^{m\times n}$ be such that $\mathcal{R}(X) \perp \mathcal{R}(Y)$.