Minimax and shrinkage properties of the least-squares estimator

[3/33%]
Show
LaTeX

In the linear model (3.4.4), show that Σaiξ^i\Sigma a_{i} \hat{\xi }_{i} (in the notation of Theorem 3.4.4) is minimax for estimating θ=Σaiξi\theta =\Sigma a_{i} \xi_{i} with squared error loss, under the restriction σ2≤M\sigma^{2} \leq M.

?

Consider the problem of estimating the mean based on X∼Nr(θ,I)X \sim N_{r}(\boldsymbol {\theta }, I), where it is thought that θi=∑j=1sβjtij\theta_{i}=\sum_{j=1}^{s} \beta_{j} t_{i}^{j} where (ti,…,tr)(t_{i}, \ldots , t_{r}) are known, (β1,…,βs)(\beta_{1}, \ldots , \beta_{s}) are unknown, and r−s>2r-s>2.

?
(a)

Find the MLE of θ\boldsymbol {\theta }, say θ^R\hat{\boldsymbol {\theta }}_{R}, if θ\boldsymbol {\theta } is assumed to be in the linear subspace

L={θ:∑j=1sβjtij=θi,i=1,…,r}. \mathcal{L}=\left\{ \boldsymbol {\theta }: \sum _{j=1}^{s} \beta _{j} t_{i}^{j}=\theta _{i}, \quad i=1, \ldots , r\right\} .
(b)

Show that L\mathcal{L} can be written in the form (6.7), and find KK.

(c)

Construct a Stein estimator that shrinks toward the MLE of part (a) and prove that it is minimax.

Consider the linear model with observed vector XX having distribution Nn(Zβ,σ2In)N_n(Z\beta ,\sigma^2I_n), where ZZ is an n×pn\times p known matrix, p<np<n, β∈Rp\beta \in \mathbb {R}^{p}, and σ2>0\sigma^2>0, and the estimation of lτβl^\tau \beta under the squared error loss, where l∈R(Z)l\in \mathcal R(Z). Show that the LSE lτβ^l^\tau \widehat{\beta } is minimax if σ2∈(0,c]\sigma^2\in (0,c] with a constant cc.

?
Remark.
?

Corresponding exercise in Shao (2003): 4.73.

References

[LeCa98] Lehmann E.L., Casella G. Theory of Point Estimation. 2nd ed., Springer, 1998.
[Sha05] Shao J. Mathematical Statistics: Exercises and Solutions. Springer, 2005.