Analysis of Variance

If the $\varepsilon_{i j}(i = 1,2, \ldots , I ; j = 1,2, \ldots , J)$ are independently distributed as $N\left(0, \sigma^{2}\right)$, prove that

are statistically independent.

Let $Y_{i j} = \mu +\alpha_{i}+\varepsilon_{i j}(i = 1,2, \ldots , I ; j = 1,2, \ldots , J)$, where $\sum_{i} d_{i} \alpha_{i} = 0 \left(\sum_{i} d_{i} \neq 0\right)$ and $E\left[\varepsilon_{i j}\right] = 0$ for all $i, j$. Using the method of Lagrange multipliers, find the least squares estimates of $\mu$ and $\alpha_{i}$. Hint: Show that the Lagrange multiplier is zero.

Let $Y_{i j k} = \mu_{i j}+\varepsilon_{i j k}$, where

say, $i = 1,2, \ldots , I ; j = 1,2, \ldots , J ; k = 1,2, \ldots , K$, and the $\varepsilon_{i j k}$ are independently distributed as $N\left(0, \sigma^{2}\right)$.

Let $Y_{i j} = \mu_{i}+\varepsilon_{i j}(i = 1,2, \ldots , I ; j = 1,2, \ldots , J)$, where the $\varepsilon_{i j}$ are independently distributed as $N\left(0, \sigma^{2}\right)$.

Suppose that we have the model

where $i = 1,2, \ldots , I ; j = 1,2, \ldots , J ; k = 1,2, \ldots , K, \sum_{i} \alpha_{i} = \sum_{j} \beta_{j} = \sum_{k} \gamma_{k} = 0$, and the $\varepsilon_{i j k}$ are independently distributed as $N\left(0, \sigma^{2}\right)$.

Consider the linear model $Y_{i j k} = \mu_{i j k}+\varepsilon_{i j k}$, where $i = 1,2, \ldots , I ; j = 1,2, \ldots , J ; k = 1,2, \ldots , K$; and the $\varepsilon_{i j k}$ are independently distributed as $N\left(0, \sigma^{2}\right)$. Let

say, where $\Delta_{i j k} = 0$ (all $i, j, k$).

Express $H: \mu_{1} = \cdots = \mu_{I}$ in the form $\mathbf{A} \mathbf{\mu } = \mathbf{0}$. What is the rank of $\mathbf{A}$? As this matrix is not unique, find another matrix, $\mathbf{A}_{1}$ say, also satisfying $H: \mathbf{A}_{1} \mathbf{\mu } = \mathbf{0}$. What would be the relationship between $\mathbf{A}$ and $\mathbf{A}_{1}$?

Obtain the identity indicated by (8.3), namely,

Obtain the least squares estimates of the $\mu_{i}$ using $\widehat{\mathbf{\beta }} = \left(\mathbf{X}^{T} \mathbf{X}\right)^{-1} \mathbf{X}^{T} \mathbf{Y}$ directly, where $\mathbf{X}$ is given by (8.2).

Prove that

Obtain a similar expression for $\sum_{i} \sum_{j}\left(Y_{i j}-\bar{Y}_{i}\right)^{2}$.

Find (a) $\mathbb {E}\left[\sum_{i} \sum_{j}\left(\bar{Y}_{i .}-\bar{Y} . .\right)^{2}\right]$ and (b) $\mathbb {E}\left[\sum_{i} \sum_{j}\left(Y_{i j}-\bar{Y}_{i .}\right)^{2}\right]$. Hint: Use Theorem 4.1(ii) for the first expression.

Prove that $H_{A B}, H_{A B 1}, H_{A B 2}$, and $H_{A B 3}$ are all equivalent to $\mu_{i j} = \mu +\alpha_{i}+\beta_{j}$.

Using the syratetric constraints, exprese the parameters $\mu , \alpha_{i}$, etc. in terms of the $\mu_{i j}$.

Obtain an $F$-test statistic for testing $H: \mu_{i j} = \mu$ in a two-way classification. Find the expected value of the numerator sum of squares of your test statistics.

Given the population means $\mu_{i j}\left(i = 1,2, \ldots , I ; j = 1,2, \ldots , J\right)$, let

and

Define $\alpha_{i} = A_{i}-\mu , \beta_{j} = B_{j}-\mu$, and

Let $Y_{i j k} = \mu +\alpha_{i}+\beta_{j}+(\alpha \beta )_{i j}+\varepsilon_{i j k}$, where $i = 1,2, \ldots , I ; j = 1,2, \ldots , J ; k = 1,2, \ldots , K_{i j}$; and the $\varepsilon_{i j k}$ are independently distributed as $N\left(0, \sigma^{2}\right)$. Given $K_{i j} = K_{i .} K_{. j} / K_{. .}$for all $i, j$, find a test statistic for testing the hypothesis $H:(\alpha \beta )_{i j} = 0$ (all $i, j$). Hint: By Exercise 2, the validity of $H$ does not depend on the weights used in the identifiability constraints $\sum_{i} u_{i} \alpha_{i} = \sum_{j} v_{j} \beta_{j} = 0$. We can therefore use $u_{i} = K_{i .} / K_{.}$ and $v_{j} = K_{. j} / K_{. .}$and find the least squares estimates of $\alpha_{i}$ and $\beta_{j}$ when $H$ is true.

Verify (8.28).

A three-factor experiment has population means $\mu_{i j k}(i = 1,2,3 ; j = 1,2,3 ; k = 1,2$), given by the following tables:

Show that the $A B C$ interactions are zero.

Let

where $\mathbf{X}$ has full rank. Suppose that the observations $\mathbf{Y}_{2}$ are missing. For the data observed, the least squares estimate of $\mathbf{\beta }$ is $\widehat{\mathbf{\beta }} = \left(\mathbf{X}_{1}^{T} \mathbf{X}_{1}\right)^{-1} \mathbf{X}_{1}^{T} \mathbf{Y}_{1}$. Let $\widehat{\mathbf{Y}}_{2} = \mathbf{X}_{2} \widehat{\mathbf{\beta }}$. Show that $\widehat{\mathbf{\beta }}$ can be obtained by applying least squares to the observed data augmented by $\widehat{\mathbf{Y}}_{2}$ and the regression matrix $\mathbf{X}$.

Suppose that we have a one-way classification

and suppose that $Y_{I J}$ is missing. Prove that the appropriate estimate of $Y_{I J}$ is the mean of the remaining observations on the mean $\mu_{I}$.

Exercises 3f, No. 2, in Section 3.7 gave the following two-stage method for finding least squares estimates for a general ANCOVA model $G$, say. First, we find the least squares estimate $\widehat{\mathbf{\beta }}$ of $\mathbf{\beta }$ for the model $\mathbb {E}\left[\mathbf{Y}\right] = \mathbf{X} \mathbf{\beta }$ and the residual sum of squares $\mathbf{Y}^{T} \mathbf{R} \mathbf{Y}$. Second, $\widehat{\gamma }_{G}$ is found by replacing $\mathbf{Y}$ by $\mathbf{Y}-\mathbf{Z} \mathbf{\gamma }$ in $\mathbf{Y}^{T} \mathbf{R Y}$ and minimizing with respect to $\mathbf{\gamma }$. This minimum value is the correct residual sum of squares for the model $G$. Third, referring to Theorem 3.6 , we see that $\widehat{\mathbf{\beta }}_{G}$ is obtained from $\widehat{\mathbf{\beta }}$ by replacing $\mathbf{Y}$ by $\mathbf{Y}-\mathbf{X} \widehat{\mathbf{\gamma }}$. This technique can be applied to the model for $H$ as well as $G$. Now apply this method to the following problem. Consider the model $Y_{i j} = \mu_{i}+\gamma_{i} x_{j}+\varepsilon_{i j}$, where $i = 1,2 ; j = 1,2, \ldots , J$; and the $\varepsilon_{i j}$ are independently distributed as $N\left(0, \sigma^{2}\right)$. Derive an $F$ statistic for testing the hypothesis that $\gamma_{1} = \gamma_{2}$ and show that this statistic is the square of the usual $t$-statistic for testing whether two lines are parallel.

Let $Y_{i j} = \mu_{i}+\gamma_{1} z_{i j}+\gamma_{2} w_{i j}+\varepsilon_{i j}$, where $i = 1,2, \ldots , I ; j = 1,2, \ldots , J$; and the $\varepsilon_{i j}$ are independently distributed as $N\left(0, \sigma^{2}\right)$.

Let $Y_{i j k} = \mu_{i j}+\gamma_{i j} z_{i j k}+\varepsilon_{i j k}$, where $i = 1,2, \ldots , I ; i = 1,2, \ldots , J$; $k = 1,2, \ldots , K$; and the $\varepsilon_{i j k}$ are independently distributed as $N\left(0, \sigma^{2}\right)$. Obtain a test statistic for testing the hypothesis