Math Problems Vault

Problem 4.7.1

(a)(b)

For the situation of Example 7.1:

?

(a)

The empirical Bayes estimator of $\boldsymbol {\theta }$ , using an unbiased estimate of $\tau^{2} /\left(\sigma^{2}+\tau^{2}\right)$ , is

\delta ^{\mathrm{EB}}=\left(1-\frac{(p-2) \sigma ^{2}}{|\mathbf{x}|^{2}}\right) \mathbf{x},

the James-Stein estimator.

(b)

The empirical Bayes estimator of $\boldsymbol {\theta }$ , using the marginal MLE of $\tau^{2} /\left(\sigma^{2}+\tau^{2}\right)$ , is

\delta ^{\mathrm{EB}}=\left(1-\frac{p \sigma ^{2}}{|\mathbf{x}|^{2}}\right)^{+} \mathbf{x},

which resembles the positive-part Stein estimator.

Problem 4.7.2

Corollary 7.2 states: let $X \sim N_{p}(\theta , \sigma^{2} I)$ , and let the estimator $\delta$ be of the form $\delta (\mathbf{x})=\mathbf{x}-g(\mathbf{x})$ , where $g(\mathbf{x})=\left\{ g_{i}(\mathbf{x})\right\}$ is differentiable. If $\mathbb {E}_{\theta }\left[\left|(\partial / \partial x_{i}) g_{i}(X)\right|\right]<\infty$ for $i=1, \ldots , p$ , then

R(\theta , \delta )=\mathbb {E}_{\theta }\left[\left|\theta -\delta (X)\right|^{2}\right]=p \sigma ^{2}+\mathbb {E}_{\theta }\left[\left|g(X)\right|^{2}\right]-2 \sigma ^{2} \sum _{i=1}^{p} \mathbb {E}_{\theta }\left[\frac{\partial }{\partial x_{i}} g_{i}(X)\right].

Establish Corollary 7.2. Be sure to verify that the conditions on $g(\mathbf{x})$ are sufficient to allow the integration-by-parts argument. [Stein $(1973,1981)$ develops these representations in the normal case.]

?

Problem 4.7.3

(a)(b)(c)

The derivation of an unbiased estimator of the risk (Corollary 7.2) can be extended to a more general model in the exponential family, the model of Corollary 3.3, where $\mathbf{X}=X_{1}, \ldots , X_{p}$ has the density

p_{\boldsymbol {\eta }}(\mathbf{x})=e^{\sum _{i=1}^{p} \eta _{i} x_{i}-A(\boldsymbol {\eta })} h(\mathbf{x}).

?

(a)

The Bayes estimator of $\boldsymbol {\eta }$ , under squared error loss, is

\mathbb {E}\left[\eta _{i} \mid \mathbf{x}\right]=\frac{\partial }{\partial x_{i}} \log m(\mathbf{x})-\frac{\partial }{\partial x_{i}} \log h(\mathbf{x}).

Show that the risk of $\mathbb {E}\left[\eta \mid \mathbf{X}\right]]$ has unbiased estimator

\sum _{i=1}^{p}\left[\frac{\partial ^{2}}{\partial x_{i}^{2}}(\log h(\mathbf{x})-2 \log m(\mathbf{x}))+\left(\frac{\partial }{\partial x_{i}} \log m(\mathbf{x})\right)^{2}\right].

[Hint: Theorem 3.5 and Problem 3.4.]

(b)

Show that the risk of the empirical Bayes estimator

\mathbb {E}\left[\eta _{i} \mid \mathbf{x}, \hat{\lambda }\right]=\frac{\partial }{\partial x_{i}} \log m(\mathbf{x} \mid \hat{\lambda }(\mathbf{x}))-\frac{\partial }{\partial x_{i}} \log h(\mathbf{x}).

of Theorem 6.3 has unbiased estimator

\sum _{i=1}^{p}\left[\frac{\partial ^{2}}{\partial x_{i}^{2}}(\log h(\mathbf{x})-2 \log m(\mathbf{x} \mid \hat{\lambda }(\mathbf{x})))+\left(\frac{\partial }{\partial x_{i}} \log m(\mathbf{x} \mid \hat{\lambda }(\mathbf{x}))\right)^{2}\right]

(c)

Use the results of part (b) to derive an unbiased estimator of the risk of the positivepart Stein estimator of (7.7.10).

Problem 4.7.4

Verify (7.7.9), the expression for the Bayes risk of $\delta^{\tau_{0}}$ . (Problem 3.12 may be helpful.)

?

Problem 4.7.5

(a)(b)

A general version of the empirical Bayes estimator (7.7.3) is given by

\delta ^{c}(\mathbf{x})=\left(1-\frac{c \sigma ^{2}}{|\mathbf{x}|^{2}}\right) \mathbf{x},

where $c$ is a positive constant.

?

(a)

Use Corollary 7.2 to verify that

\mathbb {E}_{\theta }\left[\left|\boldsymbol {\theta }-\delta ^{c}(\mathbf{X})\right|^{2}\right]=p \sigma ^{2}+c \sigma ^{4}[c-2(p-2)] \mathbb {E}_{\theta }\left[\frac{1}{|\mathbf{X}|^{2}}\right].

(b)

Show that the Bayes risk, under $\Theta \sim N_{p}\left(0, \tau^{2} I\right)$ , is given by

r\left(\pi , \delta ^{c}\right)=\sigma ^{2}\left[p+\frac{c \sigma ^{2}}{\sigma ^{2}+\tau ^{2}}\left(\frac{c}{p-2}-2\right)\right]

and is minimized by choosing $c=p-2$ .

Problem 4.7.6

(a)(b)(c)

For the model

\begin{align} \mathbf{X} \mid \boldsymbol {\theta } & \sim N_{p}\left(\boldsymbol {\theta }, \sigma ^{2} I\right), \\ \boldsymbol {\theta } \mid \tau ^{2} & \sim N_{p}\left(\mu , \tau ^{2} I\right): \end{align}

Show that:

?

(a)

The empirical Bayes estimator, using an unbiased estimator of $\tau^{2} /\left(\sigma^{2}+\tau^{2}\right)$ , is the Stein estimator

\delta _{i}^{\mathrm{JS}}(\mathbf{x})=\mu _{i}+\left(1-\frac{(p-2) \sigma ^{2}}{\Sigma \left(x_{i}-\mu _{i}\right)^{2}}\right)\left(x_{i}-\mu _{i}\right).

(b)

If $p \geq 3$ , the Bayes risk, under squared error loss, of $\delta^{\mathrm{JS}}$ is $r\left(\tau , \delta^{\mathrm{JS}}\right)=r\left(\tau , \delta^{\tau }\right)+ 2 \sigma^{4} /\left(\sigma^{2}+\tau^{2}\right)$ , where $r\left(\tau , \delta^{\tau }\right)$ is the Bayes risk of the Bayes estimator.

(c)

If $p<3$ , the Bayes risk of $\delta^{\mathrm{JS}}$ is infinite. [Hint: Show that if $Y \sim \chi_{m}^{2}, \mathbb {E}\left[1 / Y\right]< \infty \Longleftrightarrow m<3]$ .

Problem 4.7.7

For the model

\begin{align} \mathbf{X} \mid \boldsymbol {\theta } & \sim N_{p}\left(\boldsymbol {\theta }, \sigma ^{2} I\right), \\ \boldsymbol {\theta } \mid \tau ^{2} & \sim N\left(\mu , \tau ^{2} I\right) \end{align}

the Bayes risk of the ordinary Stein estimator

\delta _{i}(\mathbf{x})=\mu _{i}+\left(1-\frac{(p-2) \sigma ^{2}}{\Sigma \left(x_{i}-\mu _{i}\right)^{2}}\right)\left(x_{i}-\mu _{i}\right)

is uniformly larger than its positive-part version

\delta _{i}^{+}(\mathbf{x})=\mu _{i}+\left(1-\frac{(p-2) \sigma ^{2}}{\Sigma \left(x_{i}-\mu _{i}\right)^{2}}\right)^{+}\left(x_{i}-\mu _{i}\right).

?

Problem 4.7.8

(a)(b)(c)

Theorem 7.5 holds in greater generality than just the normal distribution. Suppose $\mathbf{X}$ is distributed according to the multivariate version of the exponential family $p_{\eta }(x)$ of (33.7),

p_{\eta }(\mathbf{x})=e^{\eta ^{\prime } \mathbf{x}-A(\eta )} h(\mathbf{x}), \quad -\infty <x_{i}<\infty ,

and a multivariate conjugate prior distribution [generalizing (3.19)] is used.

?

(a)

Show that $\mathbb {E}\left[\mathbf{X} \mid \eta \right]=\nabla A(\eta )$ .

(b)

If $\mu =0$ in the prior distribution (see 3.19), show that $r(\tau , \delta ) \geq r\left(\tau , \delta^{+}\right)$ , where $\delta (\mathbf{x})=[1-B(\mathbf{x})] \mathbf{x}$ and $\delta^{+}(\mathbf{x})=[1-B(\mathbf{x})]^{+} \mathbf{x}$ .

(c)

If $\mu \neq 0$ , the estimator $\delta (\mathbf{x})$ would be modified to $\mu +\delta (\mathbf{x}-\mu )$ . Establish a result similar to part (b) for this estimator.

Problem 4.7.9

(a)(b)

?

(a)

For the model (7.7.15), show that the marginal distribution of $X_{i}$ is negative binomial $(a, 1 / b+1)$ ; that is,

\mathbb {P}\left(X_{i}=x\right)=\binom {a+x-1}{x}\left(\frac{b}{b+1}\right)^{x}\left(\frac{1}{b+1}\right)^{a}

with $\mathbb {E}\left[X_{i}\right]=a b$ and var $X_{i}=a b(b+1)$ .

(b)

If $X_{1}, \ldots , X_{m}$ are iid according to the negative binomial distribution in part (a), show that the conditional distribution of $X_{j} \mid \sum_{1}^{m} X_{i}$ is the negative hypergeometric distribution, given by

\mathbb {P}\left(X_{j}=x \mid \sum _{1}^{m} X_{i}=t\right)=\frac{\binom {a+x-1}{x}\binom {(m-1) a+t-x-1}{t-x}}{\binom {m a+t-1}{t}}

with $\mathbb {E}\left[X_{j}\right]=t / m$ and var $X_{j}=(m-1) t(m a+t) / m^{2}(m a+1)$ .

Problem 4.7.10

(a)(b)(c)

For the situation of Example 7.6:

?

(a)

Show that the Bayes estimator under the loss $L_{k}(\lambda , \delta )$ of (7.7.16) is given by (7.7.17).

(b)

Verify (7.7.19) and (7.7.20).

(c)

Evaluate the Bayes risks $r\left(0, \delta^{1}\right)$ and $r\left(1, \delta^{0}\right)$ . Which estimator, $\delta^{0}$ or $\delta^{1}$ , is more robust?

Problem 4.7.11

For the situation of Example 7.6, evaluate the Bayes risk of the empirical Bayes estimator (7.7.20) for $k=0$ and 1. What values of the unknown hyperparameter $b$ are least and which are most favorable to the empirical Bayes estimator?

?

Problem 4.7.12

Consider a hierarchical Bayes estimator for the Poisson model (7.7.15) with loss (7.7.16). Using the distribution (5.6.27) for the hyperparameter $b$ , show that the Bayes estimator is

\left(\frac{p \bar{x}+\alpha -k}{p \bar{x}+p a+\alpha +\beta -k}\right)\left(a+x_{i}-k\right).

[Hint: Show that the Bayes estimator is $\mathbb {E}\left[\lambda^{1-k} \mid \mathbf{x}\right] / \mathbb {E}\left[\lambda^{-k} \mid \mathbf{x}\right]$ and that

\mathbb {E}\left[\lambda ^{r} \mid \mathbf{x}\right]=\frac{\Gamma (p \bar{x}+p a+\alpha +\beta ) \Gamma (p \bar{x}+\alpha +r)}{\Gamma (p \bar{x}+\alpha ) \Gamma (p \bar{x}+p a+\alpha +\beta +r)} \frac{\Gamma \left(a+x_{i}+r\right)}{\Gamma \left(a+x_{i}\right)}.

?

Problem 4.7.13

(a)(b)

Prove the following: Two matrix results that are useful in calculating estimators from multivariate hierarchical models are

?

(a)

For any vector $a$ of the form $a=\left(I-\frac{1}{s} J\right) b, \mathbf{1}^{\prime } a=\Sigma a_{i}=0$ .

(b)

If $B$ is an idempotent matrix (that is, $B^{2}=I$ ) and $a$ is a scalar, then

(I+a B)^{-1}=I-\frac{a}{1+a} B.

Problem 4.7.14

(a)(b)(c)(d)

For the situation of Example 7.7:

?

(a)

Show how to derive the empirical Bayes estimator $\delta^{L}$ of (7.7.28).

(b)

Verify the Bayes risk of $\delta^{L}$ of (7.7.29).

For the situation of Example 7.8:

(c)

Show how to derive the empirical Bayes estimator $\delta^{\mathrm{EB}_{2}}$ of (7.7.33).

(d)

Verify the Bayes risk of $\delta^{\mathrm{EB}_{2}}$ , (7.7.34).

Problem 4.7.15

(a)(b)(c)

The empirical Bayes estimator (7.7.27) can also be derived as a hierarchical Bayes estimator. Consider the hierarchical model

\begin{align} X_{i j} \mid \xi _{i} & \sim N\left(\xi _{i}, \sigma ^{2}\right), \quad j=1, \ldots , n, \quad i=1, \ldots , s, \\ \xi _{i} \mid \mu & \sim N\left(\mu , \tau ^{2}\right), \quad i=1, \ldots , s, \\ \mu & \sim \text{ Uniform }(-\infty , \infty ) \end{align}

where $\sigma^{2}$ and $\tau^{2}$ are known.

?

(a)

Show that the Bayes estimator, with respect to squared error loss, is

\mathbb {E}\left[\xi _{i} \mid \mathbf{x}\right]=\frac{\sigma ^{2}}{\sigma ^{2}+n \tau ^{2}} \mathbb {E}\left[\mu \mid \mathbf{x}\right]+\frac{n \tau ^{2}}{\sigma ^{2}+n \tau ^{2}} \bar{x}_{i}

where $\mathbb {E}\left[\mu \mid \mathbf{x}\right]$ is the posterior mean of $\mu$ .

(b)

Establish that $\mathbb {E}\left[\mu \mid \mathbf{x}\right]=\bar{x}=\Sigma x_{i j} / n s$ . [This can be done by evaluating the expectation directly, or by showing that the posterior distribution of $\xi_{i} \mid \mathbf{x}$ is

\xi _{i} \left\lvert \, \mathbf{x} \sim N\left[\frac{\sigma ^{2}}{\sigma ^{2}+n \tau ^{2}} \bar{x}+\frac{n \tau ^{2}}{\sigma ^{2}+n \tau ^{2}} \bar{x}_{i}, \frac{\sigma ^{2}}{\sigma ^{2}+n \tau ^{2}}\left(n \tau ^{2}+\frac{\sigma ^{2}}{s}\right)\right].\right.

Note that the $\xi_{i}$ 's are not independent a posteriori. In fact,

\boldsymbol {\xi } \left\lvert \, \mathbf{x} \sim N_{s}\left(\frac{n \tau ^{2}}{\sigma ^{2}+n \tau ^{2}} M, \frac{n \sigma ^{2} \tau ^{2}}{\sigma ^{2}+n \tau ^{2}} M\right)\right.,

where $\left.M=I+\left(\sigma^{2} / n \tau^{2}\right) J\right]$ .

(c)

Show that the empirical Bayes estimator (7.7.32) can also be derived as a hierarchical Bayes estimator, by appending the specification $(\alpha , \beta ) \sim$ Uniform $\left(\Re^{2}\right)$ [that is, $\pi (\alpha , \beta )=d \alpha d \beta ,-\infty <\alpha , \beta <\infty$ ] to the hierarchy (7.7.30).

Problem 4.7.16

(a)(b)(c)

Generalization of model (7.7.23) to the case of unequal $n_{i}$ is, perhaps, not as straightforward as one might expect. Consider the generalization

\begin{align} X_{i j} \mid \xi _{i} & \sim N\left(\xi _{i}, \sigma ^{2}\right), \quad j=1, \ldots , n_{i}, \quad i=1, \ldots , s, \\ \xi _{i} \mid \mu & \sim N\left(\mu , \tau _{i}^{2}\right), \quad i=1, \ldots , s. \end{align}

We also make the assumption that $\tau_{i}^{2}=\tau^{2} / n_{i}$ . Show that:

?

(a)

The above model is equivalent to

\begin{align} & \mathbf{Y} \sim N_{s}\left(\lambda , \sigma ^{2} I\right), \\ & \lambda \sim N_{s}\left(\mathbf{Z} \mu , \tau ^{2} I\right) \end{align}

where $Y_{i}=\sqrt{n_{i}} \bar{X}_{i}, \lambda_{i}=\sqrt{n_{i}} \xi_{i}$ and $\mathbf{z}=\left(\sqrt{n_{1}}, \ldots , \sqrt{n_{s}}\right)^{\prime }$ .

(b)

The Bayes estimator of $\xi_{i}$ , using squared error loss, is

\frac{\sigma ^{2}}{\sigma ^{2}+\tau ^{2}} \mu +\frac{\tau ^{2}}{\sigma ^{2}+\tau ^{2}} \bar{x}_{i}.

(c)

The marginal distribution of $Y_{i}$ is $Y_{i} \sim N_{s}\left(\mathbf{Z} \mu ,\left(\sigma^{2}+\tau^{2}\right) I\right)$ , and an empirical Bayes estimator of $\xi$ is

\delta _{i}^{E B}=\bar{x}+\left(1-\frac{(s-3) \sigma ^{2}}{\Sigma n_{i}\left(\bar{x}_{i}-\bar{x}\right)^{2}}\right)\left(\bar{x}_{i}-\bar{x}\right)

where $\bar{x}_{i}=\Sigma_{j} x_{i j} / n_{i}$ and $\bar{x}=\Sigma_{i} n_{i} \bar{x}_{i} / \Sigma_{i} n_{i}$ . [Without the assumption that $\tau_{i}^{2}=\tau^{2} / n_{i}$ , one cannot get a simple empirical Bayes estimator. If $\tau_{i}^{2}=\tau^{2}$ , the likelihood estimation can be used to get an estimate of $\tau^{2}$ to be used in the empirical Bayes estimator. This is discussed by Morris (1983a).]

Problem 4.7.17

(a)(b)(c)(d)

(Empirical Bayes estimation in a general case). A general version of the hierarchical models of Examples 7.7 and 7.8 is

\begin{align} & \mathbf{X} \mid \xi \sim N_{s}\left(\xi , \sigma ^{2} I\right), \\ & \xi \mid \boldsymbol {\beta } \sim N_{s}\left(\mathbf{Z} \boldsymbol {\beta }, \tau ^{2} I\right) \end{align}

where $\sigma^{2}$ and $\mathbf{Z}_{s \times r}$ , of rank $r$ , are known and $\tau^{2}$ and $\boldsymbol {\beta }_{r \times 1}$ are unknown. Under this model show that:

?

(a)

The Bayes estimator of $\xi$ , under squared error loss, is

\mathbb {E}\left[\boldsymbol {\xi } \mid \mathbf{x}, \boldsymbol {\beta }\right]=\frac{\sigma ^{2}}{\sigma ^{2}+\tau ^{2}} z \boldsymbol {\beta }+\frac{\tau ^{2}}{\sigma ^{2}+\tau ^{2}} \mathbf{x}.

(b)

Marginally, the distribution of $\mathbf{X} \mid \boldsymbol {\beta }$ is $\mathbf{X} \mid \boldsymbol {\beta } \sim N_{s}\left(\mathbf{Z} \boldsymbol {\beta },\left(\sigma^{2}+\tau^{2}\right) I\right)$ .

(c)

Under the marginal distribution in part (b),

\begin{align} \mathbb {E}\left[\left(\mathbf{Z}^{\prime } \mathbf{Z}\right)^{-1} \mathbf{Z}^{\prime } \mathbf{x}\right] & =\mathbb {E}\left[\hat{\boldsymbol {\beta }}\right]=\boldsymbol {\beta }, \\ \mathbb {E}\left[\frac{s-r-2}{|\mathbf{X}-\mathbf{Z} \hat{\boldsymbol {\beta }}|^{2}}\right] & =\frac{1}{\sigma ^{2}+\tau ^{2}}, \end{align}

and, hence, an empirical Bayes estimator of $\xi$ is

\delta ^{E B}=\mathbf{Z} \hat{\boldsymbol {\beta }}+\left(1-\frac{(s-r-2) \sigma ^{2}}{|\mathbf{x}-\mathbf{Z} \hat{\boldsymbol {\beta }}|^{2}}\right)(\mathbf{x}-\mathbf{Z} \hat{\boldsymbol {\beta }}).

(d)

The Bayes risk of $\delta^{E B}$ is $r\left(\tau , \delta^{\tau }\right)+(r+2) \sigma^{4} /\left(\sigma^{2}+\tau^{2}\right)$ , where $r\left(\tau , \delta^{\tau }\right)$ is the risk of the Bayes estimator.

Problem 4.7.18

(a)(b)(c)

In a manner similar to the previous problem, we can derive hierarchical Bayes estimators for the model

\begin{align} \mathbf{X} \mid \boldsymbol {\xi } & \sim N_{s}\left(\boldsymbol {\xi }, \sigma ^{2} I\right), \\ \boldsymbol {\xi } \mid \boldsymbol {\beta } & \sim N_{s}\left(\mathbf{Z} \boldsymbol {\beta }, \tau ^{2} I\right), \\ \boldsymbol {\beta } & \sim \operatorname {Uniform}\left(\Re ^{r}\right) \end{align}

where $\sigma^{2}$ and $\mathbf{Z}_{s \times r}$ , of rank $r$ , are known and $\tau^{2}$ is unknown.

?

(a)

The prior distribution of $\boldsymbol {\xi }$ , unconditional on $\beta$ , is proportional to

\pi (\xi )=\int _{\Re ^{r}} \pi (\xi \mid \beta ) d \beta \propto e^{-\frac{1}{2} \frac{\xi ^{\prime }(I-H) \xi }{\tau ^{2}}},

where $H=\mathbf{Z}\left(\mathbf{Z}^{\prime } \mathbf{Z}\right)^{-1} \mathbf{Z}^{\prime }$ projects from $\Re^{s}$ to $\Re^{r}$ . [Hint: Establish that

\begin{align} (\xi -\mathbf{Z} \boldsymbol {\beta })^{\prime }(\boldsymbol {\xi }-\mathbf{Z} \boldsymbol {\beta })=\xi ^{\prime }(I-H) \boldsymbol {\xi } & \\ & +\left[\boldsymbol {\beta }-\left(\mathbf{Z}^{\prime } \mathbf{Z}\right)^{-1} \mathbf{Z}^{\prime } \boldsymbol {\xi }\right]^{\prime } \mathbf{Z}^{\prime } \mathbf{Z}\left[\boldsymbol {\beta }-\left(\mathbf{Z}^{\prime } \mathbf{Z}\right)^{-1} \mathbf{Z}^{\prime } \boldsymbol {\xi }\right] \end{align}

to perform the integration on $\boldsymbol {\beta }$ .]

(b)

Show that

\boldsymbol {\xi } \left\lvert \, \mathbf{x} \sim N_{s}\left(\frac{\tau ^{2}}{\sigma ^{2}+\tau ^{2}} M, \frac{\sigma ^{2} \tau ^{2}}{\sigma ^{2}+\tau ^{2}} M\right)\right.

where $M=I+\left(\sigma^{2} / \tau^{2}\right) H$ , and hence that the Bayes estimator is given by

\frac{\sigma ^{2}}{\sigma ^{2}+\tau ^{2}} H \mathbf{x}+\frac{\tau ^{2}}{\sigma ^{2}+\tau ^{2}} \mathbf{x}

where $\mathbf{Z} \hat{\boldsymbol {\beta }}=H \mathbf{x}$ . [Hint: Establish that

\begin{align} & \frac{1}{\tau ^{2}} \xi ^{\prime }(I-H) \xi +\frac{1}{\sigma ^{2}}(\mathbf{x}-\xi )^{\prime }(\mathbf{x}-\xi ) \\ & = \frac{\sigma ^{2}+\tau ^{2}}{\sigma ^{2} \tau ^{2}}\left[\left(\xi -\frac{\tau ^{2}}{\sigma ^{2}+\tau ^{2}} M \mathbf{x}\right)^{\prime } M^{-1}\left(\xi -\frac{\tau ^{2}}{\sigma ^{2}+\tau ^{2}} M \mathbf{x}\right)\right] \\ & +\frac{1}{\sigma ^{2}+\tau ^{2}} \mathbf{x}^{\prime }(I-H) \mathbf{x} \end{align}

where $M^{-1}=I-\frac{\sigma^{2}}{\sigma^{2}+\tau^{2}} H$ .]

(c)

Marginally, $\mathbf{X}^{\prime }(I-H) \mathbf{X} \sim \left(\sigma^{2}+\tau^{2}\right) \chi_{s-r}^{2}$ . This leads us to the empirical Bayes estimator

H \mathbf{x}+\left(1-\frac{(s-r-2) \sigma ^{2}}{\mathbf{x}^{\prime }(I-H) \mathbf{x}}\right)(\mathbf{x}-H \mathbf{x})

which is equal to the empirical Bayes estimator of Problem 7.17(c).

[The model in this and the previous problem can be substantially generalized. For example, both $\sigma^{2} I$ and $\tau^{2} I$ can be replaced by full, positive definite matrices. At the cost of an increase in the complexity of the matrix calculations and the loss of simple answers, hierarchical and empirical Bayes estimators can be computed. The covariances, either scalar or matrix, can also be unknown, and inverted gamma (or inverted Wishart) prior distributions can be accommodated. Calculations can be implemented via the Gibbs sampler. Note that these generalizations encompass the "unequal $n_{i}$ " case (see Problem 7.16), but there are no simple solutions for this case. Many of these estimators also possess a minimax property, which will be discussed in Chapter 5.]

Problem 4.7.19

As noted by Morris (1983a), an analysis of variance-type hierarchical model, with unequal $n_{i}$ , will yield closed-form empirical Bayes estimators if the prior variances are proportional to the sampling variances. Show that, for the model

\begin{align} X_{i j} \mid \xi _{i} & \sim N\left(\xi _{i}, \sigma ^{2}\right), \quad j=1, \ldots , n_{i}, \quad i=1, \ldots , s \\ \boldsymbol {\xi } \mid \boldsymbol {\beta } & \sim N_{s}\left(\mathbf{Z} \boldsymbol {\beta }, \tau ^{2} D^{-1}\right) \end{align}

where $\sigma^{2}$ and $\mathbf{Z}_{s \times r}$ , of full rank $r$ , are known, $\tau^{2}$ is unknown, and $D=\operatorname {diag}\left(n_{1}, \ldots , n_{s}\right)$ , an empirical Bayes estimator is given by

\delta ^{\mathrm{EB}}=\mathbf{Z} \hat{\boldsymbol {\beta }}+\left(1-\frac{(s-r-2) \sigma ^{2}}{(\overline{\mathbf{x}}-\mathbf{Z} \hat{\boldsymbol {\beta }})^{\prime } D(\overline{\mathbf{x}}-\mathbf{Z} \hat{\boldsymbol {\beta }})}\right)(\overline{\mathbf{x}}-\mathbf{Z} \hat{\boldsymbol {\beta }})

with $\overline{\mathbf{x}}_{i}=\Sigma_{j} x_{i j} / n_{i}, \overline{\mathbf{x}}=\left\{ \bar{x}_{i}\right\}$ , and $\hat{\boldsymbol {\beta }}=\left(\mathbf{Z}^{\prime } D \mathbf{Z}\right)^{-1} \mathbf{Z}^{\prime } D \overline{\mathbf{x}}$ .

?

Problem 4.7.20

(a)(b)(c)(d)

An entertaining (and unjustifiable) result which abuses a hierarchical Bayes calculation yields the following derivation of the James-Stein estimator. Let $X \sim N_{p}(\boldsymbol {\theta }, I)$ and $\boldsymbol {\theta } \mid \tau^{2} \sim N_{p}\left(0, \tau^{2} I\right)$ .

?

(a)

Verify that conditional on $\tau^{2}$ , the posterior and marginal distributions are given by

\begin{align} \pi \left(\boldsymbol {\theta } \mid \mathbf{x}, \tau ^{2}\right) & =N_{p}\left(\frac{\tau ^{2}}{\tau ^{2}+1} \mathbf{x}, \frac{\tau ^{2}}{\tau ^{2}+1} I\right) \\ m\left(\mathbf{x} \mid \tau ^{2}\right) & =N_{p}\left[0,\left(\tau ^{2}+1\right) I\right] \end{align}

(b)

Show that, taking $\pi \left(\tau^{2}\right)=1,-1<\tau^{2}<\infty$ , we have

\begin{align} & \iint _{\Re p} \boldsymbol {\theta } \pi \left(\boldsymbol {\theta } \mid \mathbf{x}, \tau ^{2}\right) m\left(\mathbf{x} \mid \tau ^{2}\right) d \boldsymbol {\theta } d \tau ^{2} \\ & =\frac{\mathbf{x}}{(2 \pi )^{p / 2}\left(|\mathbf{x}|^{2}\right)^{p / 2-1}}\left[\Gamma \left(\frac{p-2}{2}\right) 2^{(p-2) / 2}-\frac{\Gamma (p / 2) 2^{p / 2}}{|\mathbf{x}|^{2}}\right] \end{align}

and

\begin{align} & \iint _{\Re p} \pi \left(\boldsymbol {\theta } \mid \mathbf{x}, \tau ^{2}\right) m\left(\mathbf{x} \mid \tau ^{2}\right) d \boldsymbol {\theta } d \tau ^{2} \\ & =\frac{1}{(2 \pi )^{p / 2}\left(|\mathbf{x}|^{2}\right)^{p / 2-1}} \Gamma \left(\frac{p-2}{2}\right) 2^{(p-2) / 2} \end{align}

and hence

\mathbb {E}\left[\boldsymbol {\theta } \mid \mathbf{x}\right]=\left(1-\frac{p-2}{|\mathbf{x}|^{2}}\right) \mathbf{x}.

(c)

Explain some implications of the result in part (b) and, why it cannot be true. [Try to reconcile it with (3.3.12).]

(d)

Why are the calculations in part (b) unjustified?