Section 5.14

Why Least Squares?

[1/100%]
Show
LaTeX
Problem 5.14.1

For a matrix Zm×n=[zij]Z_{m\times n} = [z_{ij}] of random variables, E[Z]E[Z] is the m×nm\times n matrix whose (i,j)(i,j)-entry is E[zij]E[z_{ij}]. Consider the standard linear model y=Xm×nβ+εy = X_{m\times n}\beta + \varepsilon such that rk⁡(X)=n\operatorname {rk}\left(X\right)=n, E[ε]=0E[\varepsilon ]=0, Cov⁡[ε]=σ2I\operatorname {Cov}[\varepsilon ] = \sigma^{2}I, and let e^\hat{e} denote the vector of random variables defined by e^=y−Xβ^\hat{e} = y - X\hat{\beta }, in which β^=(XTX)−1XTy=X†y\hat{\beta } = (X^{T}X)^{-1}X^{T}y = X^{\dagger }y. Demonstrate that

σ^2=e^Te^m−n \hat{\sigma }^{2} = \frac{\hat{e}^{T}\hat{e}}{m-n}

is an unbiased estimator for σ2\sigma^{2}.

?
Remark.
?

(5.14.5): under the standard linear model, μyi=Xi∗β\mu_{y_i} = X_{i*}\beta and Cov⁡[yi,yj]=σ2\operatorname {Cov}[y_i,y_j] = \sigma^{2} if i=ji=j, 00 otherwise. Exercise 5.9.13: rk⁡(P)=trace⁡(P)\operatorname {rk}\left(P\right) = \operatorname {trace}(P) for any projector PP.