Estimation in Multicollinear Models

Derive formulas (13.11) - (13.13) for computing the mean-square error of an arbitrary estimate $\tilde{\theta }_{n}$ of the vector $\theta$.

Show that for an arbitrary ridge estimate $\bar{\theta }_{n}(k)$ the quantities $l_{n}(k)$ and $d_{n}(k)$ are determined by relations (13.16) and (13.17), respectively.

Show that for $k = 0$ we have $\bar{\theta }_{n}(k) = \widehat{\theta }_{n}, l_{n}(k) = 0, d_{n}(k) = \sigma^{2} \operatorname {tr}\left[W_{n}^{-1}\right]$.

Consider the limiting values of the ridge estimate $\bar{\theta }_{n}(k)$ and its characteristics $l_{n}(k), d_{n}(k)$ and $\Delta_{n}(k)$ as $k \rightarrow \infty$. What is the consequence of using too large values of the regularization parameter $k$ in the ridge estimation algorithm?

Table 13.1 gives the results of observations in a model with two parameters $\theta_{1}$ and $\theta_{2}$:

where $n = 10$, $\varepsilon_{m} \sim \mathcal{N}\left(0 ; 0,25\right)$, and the errors $\varepsilon_{m}$ and $\varepsilon_{i}$ are independent for $m \neq i$; the fourth column of Table 13.1 gives the exact values of the measured dependence $z_{m}^{(0)} = h_{1,m} \theta_{1}^{(0)}+h_{2,m} \theta_{2}^{(0)}$, where $\theta_{1}^{(0)}$ and $\theta_{2}^{(0)}$ are the true (unknown) values of the parameters $\theta_{1}$ and $\theta_{2}$.

Table 13.1

Using the data of Table 13.1, construct $\bar{\theta }_{n}(k)$ and $\Delta_{n}(k)$ for $k = 0,05$, $k = 0,5$ and $k = 5$. Choose the best of these estimates by the criterion of minimum mean-square error $\Delta_{n}(k)$.

Table 13.1 gives the results of observations in a model with two parameters $\theta_{1}$ and $\theta_{2}$:

where $n = 10$, $\varepsilon_{m} \sim \mathcal{N}\left(0 ; 0,25\right)$, and the errors $\varepsilon_{m}$ and $\varepsilon_{i}$ are independent for $m \neq i$; the fourth column of Table 13.1 gives the exact values of the measured dependence $z_{m}^{(0)} = h_{1,m} \theta_{1}^{(0)}+h_{2,m} \theta_{2}^{(0)}$, where $\theta_{1}^{(0)}$ and $\theta_{2}^{(0)}$ are the true (unknown) values of the parameters $\theta_{1}$ and $\theta_{2}$.

Table 13.1

Let $\tilde{\theta }_{n}$ equal $\widehat{\theta }_{n}$ or $\bar{\theta }_{n}(0,0167), \widetilde{z}_{m} = \widetilde{h}_{m} \widetilde{\theta }_{n}$ be the corresponding estimate of the exact value of the measured dependence $z_{m}^{(0)} = h_{m} \theta^{(0)}$, $m = 1, \ldots , n$. Let also $\delta_{n} = \sum_{m = 1}^{n}\left(\widetilde{z}_{m}-z_{m}^{(0)}\right)^{2}$. For which of the estimates will the quantity $\delta_{n}$ be smaller? Does the quantity $\delta_{n}$ change significantly when passing from the OLS estimate $\bar{\theta }_{n}(0)$ to the ridge estimate $\bar{\theta }_{n}(0,0167)$? Is the following statement true: least squares does not make it possible to construct ``good'' estimates of the parameters $\theta^{(0)}$, but does make it possible to estimate quite accurately the measured dependence $\left\{ z_{m}^{(0)}, m = 1, \ldots , n\right\}$?

Using the data from Table 13.3, find estimates of the parameters of the production function model $Y = F(K, L)$ if: