Prediction from a fitted regression

Show that the square of the total bias $\left\| \mathbf{\mu }-\mathbb {E}\left[\mathbf{X} \widehat{\mathbf{\beta }}\right]\right\|^{2}$ is equal to the first term in (12.8).

Consider the regression model

where $\sum_{i} x_{i} = \sum_{i} z_{i} = 0$ and $\sum_{i} x_{i}^{2} = \sum_{i} z_{i}^{2} = 1$. Suppose that we want to predict the response $Y$ corresponding to a vector $(x, z)$ using a predictor of the form

where $\tilde{\gamma }_{1}$ is the least squares estimate obtained by fitting the model

Find the form of the posterior predictive density for straight-line regression. Assume that the explanatory variable has been centered and that the prior mean is zero. Also assume that you are predicting a single value (i.e., assume that $m = 1$).

In fitting the straight line $Y_{i} = \beta_{0}+\beta_{1} x_{i}+\varepsilon_{i}(i = 1,2, \ldots , n)$, prove that $\bar{Y}$ and $\widehat{\beta }_{1}$ are uncorrelated. If $\widehat{Y}_{0} = \widehat{\beta }_{0}+\widehat{\beta }_{1} x_{0}$, deduce that

In Section 9.2.2 suppose that $\mathbf{x}_{0}^{T} = \left(\mathbf{x}_{10}^{T}, \mathbf{x}_{20}^{T}\right)$, where $\mathbf{x}_{0}$ is $p \times 1$ and $\mathbf{x}_{10}$ is $k \times 1$. The prediction at $\mathbf{x}_{0}$ is $\widehat{Y}_{0} = \mathbf{x}_{0}^{T} \widehat{\mathbf{\beta }}$, whereas the correct prediction is $\widehat{Y}_{10} = \mathbf{x}_{10}^{T} \widehat{\mathbf{\beta }}_{1}$. Compare $\widehat{Y}_{0}$ and $\widehat{Y}_{10}$ with respect to their means and variances.

Given the predictor $\widehat{Y} = \widehat{\beta }_{0}+\widehat{\beta }_{1} x_{1}+\cdots +\widehat{\beta }_{p-1} x_{p-1}$, show that $\widehat{Y}$ has a minimum variance of $\sigma^{2} / n$ at the $x$ point $x_{j} = \bar{x}_{. j}(j = 1,2, \ldots , p-1)$. Hint: Consider the model

Generalize the argument given in Section 5.4; namely, show that the addition of several regressors to a regression model cannot decrease the variance of the prediction $\widehat{Y}$.