Matrix Differentiation

Using the result of Part (c) of Exercise 6.2, verify that every neighborhood of a point $x$ in $\mathbb {R}^{m \times 1}$ is an open set.

Let $f$ represent a function, defined on a set $S$, of a vector $x = (x_1, \ldots , x_m)'$ of $m$ variables, suppose that the set $S$ contains at least some interior points, and let $c$ represent an arbitrary one of those points. Verify that if $f$ is $k$ times continuously differentiable at $c$, then $f$ is $k$ times continuously differentiable at every point in some neighborhood of $c$.

Let $X = \{ x_{ij}\}$ represent an $m \times n$ matrix of $mn$ variables, and let $x$ represent an $mn$-dimensional column vector obtained by rearranging the elements of $X$ (in the form of a column vector). Further, let $S$ represent a set of $X$-values, and let $S^{*}$ represent the corresponding set of $x$-values (i.e., the set obtained by rearranging the elements of each $m \times n$ matrix in $S$ in the form of a column vector). Verify that an $mn$-dimensional column vector is an interior point of $S^{*}$ if and only if it is a rearrangement of an $m \times n$ matrix that is an interior point of $S$.

Let $f$ represent a function whose domain is a set $S$ in $\mathbb {R}^{m \times 1}$ (that contains at least some interior points). Show that the Hessian matrix $Hf$ of $f$ is the gradient matrix of the gradient vector $(Df)'$ of $f$.

Let $g$ represent a function, defined on a set $S$, of a vector $x = (x_1, \ldots , x_m)'$ of $m$ variables, let $S^{*} = \{ x \in S : g(x) \neq 0\}$, and let $c$ represent any interior point of $S^{*}$ at which $g$ is continuously differentiable. Show that (for any positive integer $k$) $g^{-k}$ is continuously differentiable at $c$ and

Let $F$ represent a $p \times p$ matrix of functions, defined on a set $S$, of a vector $x = (x_1, \ldots , x_m)'$ of $m$ variables. Let $c$ represent any interior point of $S$ at which $F$ is continuously differentiable. Show that if $F$ is idempotent at all points in some neighborhood of $c$, then (at $x = c$)

Let $g$ represent a function, defined on a set $S$, of a vector $x = (x_1, \ldots , x_m)'$ of $m$ variables, and let $f$ represent a $p \times 1$ vector of functions (defined on $S$) of $x$. Let $c$ represent any interior point (of $S$) at which $g$ and $f$ are continuously differentiable. Show that $gf$ is continuously differentiable at $c$ and that (at $x = c$)

Let $X = \{ x_{ij}\}$ represent an $m \times n$ matrix of ``independent'' variables, and suppose that $X$ is free to range over all of $\mathbb {R}^{m \times n}$. Show that, for $k = 2, 3, \ldots ,$

Let $X = \{ x_{st}\}$ represent an $m \times n$ matrix of ``independent'' variables, and suppose that $X$ is free to range over all of $\mathbb {R}^{m \times n}$.

Let $X = \{ x_{ij}\}$ represent an $m \times m$ matrix. Let $f$ represent a function of $X$ defined on a set $S$ comprising some or all $m \times m$ symmetric matrices. Suppose that, for purposes of differentiation, $f$ is to be interpreted as a function of the $[m(m+1)/2]$-dimensional column vector $x$ whose elements are $x_{ij}$ $(j \leq i = 1, \ldots , m)$. Suppose further that there exists a function $g$, whose domain is a set $T$ of not-necessarily-symmetric matrices that contains $S$ as a proper subset, such that $g(X) = f(X)$ for $X \in S$, so that $g$ is a function of $X$ and $f$ is the function obtained by restricting the domain of $g$ to $S$. Define $S^{*} = \{ x : X \in S\}$. Let $c$ represent an interior point of $S^{*}$, and let $C$ represent the corresponding value of $X$. Show that if $C$ is an interior point of $T$ and if $g$ is continuously differentiable at $C$, then $f$ is continuously differentiable at $c$ and that (at $x = c$)

Let $h = \{ h_i\}$ represent an $n \times 1$ vector of functions, defined on a set $S$, of a vector $x = (x_1, \ldots , x_m)'$ of $m$ variables. Let $g$ represent a function, defined on a set $T$, of a vector $y = (y_1, \ldots , y_n)'$ of $n$ variables. Suppose that $h(x) \in T$ for every $x$ in $S$, and take $f$ to be the composite function defined (on $S$) by $f(x) = g[h(x)]$. Show that if $h$ is twice continuously differentiable at an interior point $c$ of $S$ and if [assuming that $h(c)$ is an interior point of $T$] $g$ is twice continuously differentiable at $h(c)$, then $f$ is twice continuously differentiable at $c$ and

Let $X = \{ x_{ij}\}$ represent an $m \times m$ matrix of $m^2$ ``independent'' variables (where $m \geq 2$), and suppose that the range of $X$ comprises all of $\mathbb {R}^{m \times m}$. Show that (for any positive integer $k$) the function $f$ defined (on $\mathbb {R}^{m \times m}$) by $f(X) = \left|X\right|^k$ is continuously differentiable at every $X$ and that

Let $F = \{ f_{is}\}$ represent a $p \times p$ matrix of functions, defined on a set $S$, of a vector $x = (x_1, \ldots , x_m)'$ of $m$ variables. Let $c$ represent any interior point (of $S$) at which $F$ is continuously differentiable. Use the results of Exercise 13.10 to show that

Let $X = \{ x_{st}\}$ represent an $m \times n$ matrix of ``independent'' variables, let $A$ represent an $m \times m$ matrix of constants, and suppose that the range of $X$ is a set $S$ comprising some or all $X$-values for which $\det (X'AX) > 0$. Show that $\log \det (X'AX)$ is continuously differentiable at any interior point $C$ of $S$ and that (at $X = C$)

Let $F = \{ f_{is}\}$ represent a $p \times p$ matrix of functions, defined on a set $S$, of a vector $x = (x_1, \ldots , x_m)'$ of $m$ variables, and let $A$ and $B$ represent $q \times p$ and $p \times q$ matrices of constants. Suppose that $S$ is the set of all $x$-values for which $F(x)$ is nonsingular and $\operatorname {det}\left(AF^{-1}(x)B\right) > 0$ or is a subset of that set. Show that if $F$ is continuously differentiable at an interior point $c$ of $S$, then $\log \det (AF^{-1}B)$ is continuously differentiable at $c$ and (at $x = c$)

Let $A$ and $B$ represent $q \times m$ and $m \times q$ matrices of constants.

Let $F = \{ f_{is}\}$ represent a $p \times p$ matrix of functions, defined on a set $S$, of a vector $x = (x_1, \ldots , x_m)'$ of $m$ variables. Let $c$ represent any interior point (of $S$) at which $F$ is continuously differentiable. By, for instance, using the result of Part (b) of Exercise 13.10, show that if $\operatorname {rk}\left(F(c)\right) \leq p-3$, then

Let $X$ represent an $m \times m$ matrix of $m^2$ ``independent'' variables. Suppose that the range of $X$ is a set $S$ comprising some or all $X$-values for which $X$ is nonsingular, and let $C$ represent an interior point of $S$. Denote the $ij$th element of $X^{-1}$ by $y_{ij}$, the $j$th column of $X^{-1}$ by $y_j$, and the $i$th row of $X^{-1}$ by $z_i'$.

Let $F = \{ f_{is}\}$ represent a $p \times p$ matrix of functions, defined on a set $S$, of a vector $x = (x_1, \ldots , x_m)'$ of $m$ variables. For any nonempty set $T = \{ t_1, \ldots , t_s\}$, whose members are integers between 1 and $m$, inclusive, define $D(T) = \partial^s F/\partial x_{t_1} \cdots \partial x_{t_s}$. Let $k$ represent a positive integer and, for $i = 1, \ldots , k$, let $j_i$ represent an arbitrary integer between 1 and $m$, inclusive.

Let $X = \{ x_{ij}\}$ represent an $m \times m$ symmetric matrix, and let $x$ represent the $m(m+1)/2$-dimensional column vector whose elements are $x_{ij}$ $(j \leq i = 1, \ldots , m)$. Define $S$ to be the set of all $x$-values for which $X$ is nonsingular and $S^{*}$ to be the set of all $x$-values for which $X$ is positive definite. Show that $S$ and $S^{*}$ are both open sets.

Let $X$ represent an $n \times p$ matrix of constants, and let $W$ represent an $n \times n$ symmetric positive definite matrix whose elements are functions, defined on a set $S$, of a vector $z = (z_1, \ldots , z_m)'$ of $m$ variables. Further, let $c$ represent any interior point (of $S$) at which $W$ is twice continuously differentiable. Show that $W - WP_{X,W}$ is twice continuously differentiable at $c$ and that (at $z = c$)

Let $X$ represent an $n \times p$ matrix and $W$ an $n \times n$ symmetric positive definite matrix, and suppose that the elements of $X$ and $W$ are functions, defined on a set $S$, of a vector $z = (z_1, \ldots , z_m)'$ of $m$ variables. And, let $c$ represent any interior point (of $S$) at which $W$ and $X$ are continuously differentiable, and suppose that $X$ has constant rank on some neighborhood of $c$. Further, let $B$ represent any $p \times n$ matrix such that $X'WXB = X'W$. Then, at $z = c$,