12.7 Effect of Model Selection on Inference: 12.7.1 Conditional and Unconditional Distributions, 12.7.2 Bias, 12.7.3 Conditional Means and Variances, 12.7.4 Estimating Coefficients Using Conditional Likelihood, 12.7.5 Other Effects of Model Selection

Consider the regression model

where $\sum_{i} x_{i} = 0$ and $\sum_{i} x_{i}^{2} = 1$. Let $r = \sum_{i} x_{i} Y_{i}$. Suppose that we estimate the parameter $\beta$ by the estimate

where $c$ is some positive constant. How does the MSE of $\tilde{\beta }$ as an estimate of $\beta$ compare with that of the usual LSE?

Suppose that you use forward selection to choose either the null model $Y_{i} = \alpha +\varepsilon_{i}$ or the simple linear regression model in Exercise 1 above. Explain how you would evaluate the conditional density (12.74) in this situation.

For any pair of random vectors $\mathbf{Y}$ and $\mathbf{Z}$, show that the difference $\operatorname {Var}\left[\mathbf{Y}\right]-\mathbb {E}\left[\operatorname {Var}\left[\mathbf{Y} \mid \mathbf{Z}\right]\right]$ is positive semidefinite. Hint: Show that the difference is $\operatorname {Var}\left[\mathbb {E}\left[\mathbf{Y} \mid \mathbf{Z}\right]\right]$.