Section 3g

3.8 Estimation with Linear Restrictions: 3.8.1 Method of Lagrange Multipliers, 3.8.2 Method of Orthogonal Projections

[5/100%]
Show
LaTeX
Problem 3g.1
?
(a)

Find the least squares estimates of α\alpha and β\beta in Example 3.5 using the two approaches described there. What is the least squares estimate of γ\gamma?

(b)

Suppose that a further constraint is introduced: namely, α=β\alpha = \beta. Find the least squares estimates for this new situation using both methods.

Problem 3g.2

By considering the identity Y−Y^H=Y−Y^+Y^−Y^H\mathbf{Y}-\widehat{\mathbf{Y}}_{H} = \mathbf{Y}-\widehat{\mathbf{Y}}+\widehat{\mathbf{Y}}-\widehat{\mathbf{Y}}_{H}, prove that

∥Y−Y^H∥2=∥Y−Y^∥2+∥Y^−Y^H∥2 \left\| \mathbf{Y}-\widehat{\mathbf{Y}}_{H}\right\| ^{2} = \left\| \mathbf{Y}-\widehat{\mathbf{Y}}\right\| ^{2}+\left\| \widehat{\mathbf{Y}}-\widehat{\mathbf{Y}}_{H}\right\| ^{2}
?
Problem 3g.3

Prove that

Var⁡[β^H]=σ2{(XTX)−1−(XTX)−1AT[A(XTX)−1AT]−1A(XTX)−1} \operatorname {Var}\left[\widehat{\beta }_{H}\right] = \sigma ^{2}\left\{ \left(\mathbf{X}^{T} \mathbf{X}\right)^{-1}-\left(\mathbf{X}^{T} \mathbf{X}\right)^{-1} \mathbf{A}^{T}\left[\mathbf{A}\left(\mathbf{X}^{T} \mathbf{X}\right)^{-1} \mathbf{A}^{T}\right]^{-1} \mathbf{A}\left(\mathbf{X}^{T} \mathbf{X}\right)^{-1}\right\}

Hence deduce that

Var⁡[β^Hj]≤Var⁡[β^j] \operatorname {Var}\left[\widehat{\beta }_{H j}\right] \leq \operatorname {Var}\left[\widehat{\beta }_{j}\right]

where β^Hj\widehat{\beta }_{H j} and β^j\widehat{\beta }_{j} are the jj th elements of β^H\widehat{\mathbf{\beta }}_{H} and β^\widehat{\mathbf{\beta }}, respectively.

?
Problem 3g.4

Show that

∥Y−Y^H∥2−∥Y−Y^∥2=σ2λ^HT(Var⁡[λ^H])−1λ^H \left\| \mathbf{Y}-\widehat{\mathbf{Y}}_{H}\right\| ^{2}-\left\| \mathbf{Y}-\widehat{\mathbf{Y}}\right\| ^{2} = \sigma ^{2} \widehat{\mathbf{\lambda }}_{H}^{T}\left(\operatorname {Var}\left[\widehat{\mathbf{\lambda }}_{H}\right]\right)^{-1} \widehat{\lambda }_{H}
?
Problem 3g.5

If X\mathbf{X} is n×pn \times p of rank⁡p\operatorname {rank} p and B\mathbf{B} is p×qp \times q of rank⁡q\operatorname {rank} q, show that rank⁡(XB)=q\operatorname {rank}(\mathbf{X B}) = q.

?