Chapter 6

Straight-Line Regression

[15/100%]

Show

LaTeX

Show LaTeX

Problem 6.1

Let $F = \widehat{\beta }_{1}^{2} \sum_{i}\left(x_{i}-\bar{x}\right)^{2} / S^{2}$ , the $F$ -statistic for testing $H: \beta_{1} = 0$ for a straight line. Using the notation of Section 6.1.5, prove that

\tilde{x}_{0}-\bar{x} = \frac{F}{F+(n-2)}\left(\widehat{x}_{0}-\bar{x}\right) .

Problem 6.2

Derive an $F$ -statistic for testing the hypothesis that two straight lines intersect at the point $(a, b)$ .

Problem 6.3

Obtain an estimate and a confidence interval for the horizontal distance between two parallel lines.

Problem 6.4

Show how to transform the following equation into a straight line so that $\alpha$ and $\beta$ can be estimated by least squares:

y = \frac{\alpha \beta }{\alpha \sin ^{2} \theta +\beta \cos ^{2} \theta } .

Section 6a6.1 The Straight Line: 6.1.1 Confidence Intervals for the Slope and Intercept, 6.1.2 Confidence Interval for the X-Intercept, 6.1.3 Prediction Intervals and Bands, 6.1.4 Prediction Intervals for the Response, 6.1.5 Inverse Prediction (Calibration)[4/100%]

Problem 6a.1

In fitting the straight line $Y_{i} = \beta_{0}+\beta_{1} x_{i}+\varepsilon_{i}(i = 1,2, \ldots , n)$ , prove that $\bar{Y}$ and $\widehat{\beta }_{1}$ are uncorrelated. If $\widehat{Y}_{0} = \widehat{\beta }_{0}+\widehat{\beta }_{1} x_{0}$ , deduce that

\operatorname {Var}\left[\widehat{Y}_{0}\right] = \sigma ^{2}\left\{ \frac{1}{n}+\frac{\left(x_{0}-\bar{x}\right)^{2}}{\sum \left(x_{i}-\bar{x}\right)^{2}}\right\}

Problem 6a.2

Using the notation of Section 6.1.2 , prove that $\widehat{\phi } = -\widehat{\beta }_{0} / \widehat{\beta }_{1}$ is the maximum likelihood estimate of $\phi$ .

Problem 6a.3

Given a general linear regression model, show how to find a confidence interval for the ratio $\mathbf{a}_{1}^{T} \mathbf{\beta } / \mathbf{a}_{2}^{T} \mathbf{\beta }$ of two linear parametric functions.

Problem 6a.4

Using the notation of Section 6.1.5, show that when $\bar{x} = 0$ ,

\frac{\widehat{x}_{0}-\tilde{x}_{0}}{\widehat{x}_{0}} = 1-r^{2}

where $r$ is the correlation coefficient of the pairs $\left(x_{i}, Y_{i}\right)$ .

Section 6b6.2 Straight Line through the Origin, 6.3 Weighted Least Squares for the Straight Line: 6.3.1 Known Weights, 6.3.2 Unknown Weights[3/100%]

Problem 6b.1

(a)(b)

Let $Y_{1}, Y_{2}, \ldots , Y_{n}$ be independent random variables such that for $i = 1,2, \ldots , n$ ,

\mathbb {E}\left[Y_{i} \mid X = x_{i}\right] = \beta _{1} x_{i}

and

\operatorname {Var}\left[Y_{i} \mid X = x_{i}\right] = \sigma ^{2} w_{i}^{-1} \quad \left(w_{i} > 0\right)

(a)

If the conditional distribution of $Y$ given $x$ is the Type III (scaled gamma) distribution,

f(y \mid x) = \frac{1}{a_{x}^{p} \Gamma (p)} y^{p-1} \exp \left(-y / a_{x}\right) \quad 0 \leq y < \infty , \quad p > 0

where $\alpha_{x}$ a function of $x$ , and $w_{i}^{-1} = x_{i}^{2}$ , prove that the maximum likeithood estimate of $\beta_{1}$ is also the weighted least squares estimate.

(b)

If the conditional distribution of $Y$ given $x$ is Poisson, and $w_{i}^{-1} = x_{i}$ , show that the maximum likelihood estimate is just the weighted least squares estimate.

Problem 6b.2

Given the model $Y_{i} = \beta_{1} x_{i}+\varepsilon_{i}(i = 1,2, \ldots , n)$ , where the $\varepsilon_{i}$ are independently distributed as $N\left(0, \sigma^{2} w_{i}^{-1}\right), w_{i} > 0$ , show how to predict $x_{0}$ for a given value $Y_{0}$ of $Y$ . Describe briefly a method for constructing a confidence interval for $x_{0}$ .

Problem 6b.3

Given the regression line

Y_{i} = \beta _{0}+\beta _{1} x_{i}+\varepsilon _{i} \quad (i = 1,2, \ldots , n)

where the $\varepsilon_{i}$ are independent with $\mathbb {E}\left[\varepsilon_{i}\right] = 0$ and $\operatorname {Var}\left[\varepsilon_{i}\right] = \sigma^{2} x_{i}^{2}$ , show that weighted least squares estimation is equivalent to ordinary least squares estimation for the model

\frac{Y_{i}}{x_{i}} = \beta _{1}+\frac{\beta _{0}}{x_{i}}+\delta _{i} .

Section 6c6.4 Comparing Straight Lines: 6.4.1 General Model, 6.4.2 Use of Dummy Explanatory Variables[4/100%]

Problem 6c.1

Using the notation of Example 6.2, prove that the least squares estimates of $\alpha_{k}$ and the common slope $\beta$ (under the null hypothesis of parallelism) are given by

\tilde{\alpha }_{k} = \bar{Y}_{k}-\tilde{\beta } \bar{x}_{k}

and

\tilde{\beta } = \frac{\sum _{k} \sum _{i}\left(Y_{k i}-\bar{Y}_{k .}\right)\left(x_{k i}-\bar{x}_{k .}\right)}{\sum _{k} \sum _{i}\left(x_{k i}-\bar{x}_{k .}\right)^{2}}

Prove that

\mathrm{RSS}_{H_{3}} = \sum _{k} \sum _{i}\left(Y_{k i}-\bar{Y}_{k} \cdot \right)^{2}-\tilde{\beta }^{2} \sum _{k} \sum _{i}\left(x_{k i}-\bar{x}_{k} \cdot \right)^{2} .

If $\widehat{\beta }_{k}$ is the least squares estimate of $\beta_{k}$ under the general model, prove that

\mathrm{RSS}_{H_{1}}-\mathrm{RSS} = \sum _{k} \widehat{\beta }_{k}^{2} \sum _{i}\left(x_{k i}-\bar{x}_{k} .\right)^{2}-\tilde{\beta }^{2} \sum _{k} \sum _{i}\left(x_{k i}-\bar{x}_{k} .\right)^{2}

Problem 6c.2

In Example 6.3 find the least squares estimates of $\alpha$ and $\beta$ when $H_{2}$ is true.

Problem 6c.3

(a)(b)

In Example 6.4 derive the following results.

(a)

Show that the least squares estimates of $\alpha$ and $\beta_{k}$ , under $H_{3}$ , are given by

\begin{aligned} & \left(N-\frac{x_{1.}^{2}}{\sum _{i} x_{1 i}^{2}}-\cdots -\frac{x_{K .}^{2}}{\sum _{i} x_{K i}^{2}}\right) \alpha ^{T} \\ & \quad \quad = \left(Y . .-\frac{x_{1 \cdot } \cdot \sum _{i} Y_{1 i} x_{1 i}}{\sum _{i} x_{1 i}^{2}}-\cdots -\frac{x_{K \cdot } \cdot \sum _{i} Y_{K i} x_{K i}}{\sum _{i} x_{K i}^{2}}\right) \end{aligned}

and

\beta _{k}^{T} = \frac{\sum _{i}\left(Y_{k i}-\alpha ^{T}\right) x_{k i}}{\sum _{i} x_{k i}^{2}} \quad (k = 1,2, \ldots , K)

(b)

When the values of $x$ are the same for each line so that $n_{k} = n$ and $x_{k i} = x_{i}(k = 1,2, \ldots , K)$ , prove that

\begin{gathered} \alpha ^{T} = \bar{Y} . .-\frac{\bar{x} \sum \sum Y_{k i}\left(x_{i}-\bar{x}\right)}{K \sum \left(x_{i}-\bar{x}\right)^{2}} = \bar{Y} . .-\bar{x} \frac{\sum _{k} \widehat{\beta }_{k}}{K}, \\ \operatorname {Var}\left[\alpha ^{T}\right] = \frac{\sigma ^{2} \sum x_{i}^{2}}{n K \sum \left(x_{i}-\bar{x}\right)^{2}} \end{gathered}

and

\operatorname {RSS}_{H_{3}} = \sum _{k} \sum _{i} Y_{k i}^{2}-\frac{\left(\sum \sum Y_{k i} x_{i}\right)^{2}}{\sum x_{i}^{2}}-\left(\alpha ^{T}\right)^{2} K n \frac{\sum \left(x_{i}-\bar{x}\right)^{2}}{\sum x_{i}^{2}}

Hint: $\bar{Y}$ . and $\widehat{\beta }_{k}$ are uncorrelated.

Problem 6c.4

(a)(b)(c)

The examples in Section 6.4 are all special cases of the following problem.

Consider the model

G: \mathbb {E}\left[Y\right] = \mathbf{X}_{1} \mathbf{\beta }_{1}+\mathbf{X}_{2} \mathbf{\beta }_{2} = \mathbf{X} \mathbf{\beta }

where $\mathbf{X}_{i}$ is $n \times p_{i}$ of rank $p_{i}(i = 1,2)$ . We want to test the hypothesis $H$ that all the elements of $\mathbf{\beta }_{\mathbf{2}}$ are equal (to $\beta$ , i.e., $\mathbf{\beta }_{2} = \mathbf{1}_{p_{2}} \beta$ ).

(a)

If $\tilde{\beta }$ is the least squares estimate of $\beta$ under $H$ , show that

\tilde{\beta } = \frac{\mathbf{1}_{p_{2}}^{T} \mathbf{X}_{2}^{T}\left(\mathbf{I}_{n}-\mathbf{P}_{1}\right) \mathbf{Y}}{\mathbf{1}_{p_{2}}^{T} \mathbf{X}_{2}^{T}\left(\mathbf{I}_{n}-\mathbf{P}_{1}\right) \mathbf{X}_{2} \mathbf{1}_{p_{2}}}

where $\mathbf{P}_{1} = \mathbf{X}_{1}\left(\mathbf{X}_{1}^{T} \mathbf{X}_{1}\right)^{-1} \mathbf{X}_{1}^{T}$ .

(b)

If $\widehat{\mathbf{\beta }}_{\mathbf{2}}$ is the least squares estimate of $\mathbf{\beta }_{\mathbf{2}}$ under $G$ , prove that

\mathrm{RSS}_{H}-\operatorname {RSS}_{G} = \left(\widehat{\mathbf{\beta }}_{2}-\tilde{\beta } \mathbf{1}_{p_{2}}\right)^{T} \mathbf{X}_{2}^{T}\left(\mathbf{I}_{n}-\mathbf{P}_{1}\right) \mathbf{Y}

(c)

If $\widehat{\mathbf{Y}}_{G}$ is the fitted regression for $G$ , prove that

\widehat{\mathbf{Y}}_{G} = \mathbf{P}_{1} \mathbf{Y}+\left(\mathbf{I}_{n}-\mathbf{P}_{1}\right) \mathbf{X}_{2} \widehat{\mathbf{\beta }}_{2}

and

\operatorname {RSS}_{H}-\operatorname {RSS}_{G} = \left(\mathbf{1}_{p_{2}} \tilde{\beta }-\widehat{\mathbf{\beta }}_{2}\right)^{T} \mathbf{X}_{2}^{T}\left(\mathbf{I}_{n}-\mathbf{P}_{1}\right) \mathbf{X}_{2}\left(\mathbf{1}_{p_{2}} \tilde{\beta }-\widehat{\mathbf{\beta }}_{2}\right)

Hint: For (a) and (b) apply Theorem 3.6 with $\mathbf{Z}$ equal to $\mathbf{X}_{2}$ and $\mathbf{1}_{p_{2}}$ , respectively. For part (c) note that $\operatorname {RSS}_{G}-\operatorname {RSS}_{H} = \left\| \widehat{\mathbf{Y}}_{G}-\widehat{\mathbf{Y}}_{H}\right\|^{2}$ .