3.2 Properties of Least Squares Estimates

Let $Y_{i} = \beta_{0}+\beta_{1} x_{i}+\varepsilon_{i}(i = 1,2, \ldots , n)$, where $\mathbb {E}\left[\varepsilon \right] = 0$ and $\operatorname {Var}\left[\varepsilon \right] = \sigma^{2} \mathbf{I}_{n}$. Find the least squares estimates of $\beta_{0}$ and $\beta_{1}$. Prove that they are uncorrelated if and only if $\bar{x} = 0$.

In order to estimate two parameters $\theta$ and $\phi$ it is possible to make observations of three types: (a) the first type have expectation $\theta$, (b) the second type have expectation $\theta +\phi$, and (c) the third type have expectation $\theta -2 \phi$. All observations are subject to uncorrelated errors of mean zero and constant variance. If $m$ observations of type (a), $m$ observations of (b), and $n$ observations of type (c) are made, find the least squares estimates $\widehat{\theta }$ and $\widehat{\phi }$. Prove that these estimates are uncorrelated if $m = 2 n$.

Let $Y_{1}, Y_{2}, \ldots , Y_{n}$ be a random sample from $N\left(\theta , \sigma^{2}\right)$. Find the linear unbiased estimate of $\theta$ with minimum variance.

Let

where $\bar{x}_{j} = \sum_{i = 1}^{n} x_{i j} / n, \mathbb {E}\left[\varepsilon \right] = \mathbf{0}$, and $\operatorname {Var}\left[\varepsilon \right] = \sigma^{2} \mathbf{I}_{n}$. If $\widehat{\beta }_{1}$ is the least squares estimate of $\beta_{1}$, show that

where $r$ is the correlation coefficient of the $n$ pairs $\left(x_{i 1}, x_{i 2}\right)$.