Least-squares estimates and complete sufficient statistics

[2/0%]

Show

LaTeX

Show LaTeX

Problem 3.4.16 from Lehmann E.L., Casella G. "Theory of Point Estimation" [LeCa98]More problems in Section 3.4 "Normal Linear Models"

(a)(b)

(a)

Show that under assumptions (4.35), if $\xi =\theta A$ , then the least squares estimate of $\theta$ is $\mathbf{x} A\left(A A^{\prime }\right)^{-1}$ .

(b)

If $(X, A)$ is multivariate normal with all parameters unknown, show that the least squares estimator of part (a) is a function of the complete sufficient statistic and, hence, prove part (a) of Theorem 4.14.

Problem 3.4.17 from Lehmann E.L., Casella G. "Theory of Point Estimation" [LeCa98]More problems in Section 3.4 "Normal Linear Models"

(a)(b)(c)

A generalization of the order statistics, to vectors, is given by the following definition.

Definition 8.1 The $c_{j}$ -order statistics of a sample of vectors are the vectors arranged in increasing order according to their $j$ th components.

Let $\mathbf{X}_{i}, i=1, \ldots , n$ , be an iid sample of $p \times 1$ vectors, and let $X=\left(X_{1}, \ldots , X_{n}\right)$ be a $p \times n$ matrix.

Theorem 4.14 states: under assumptions (4.35), with $\xi =\theta A$ , the following hold.

(a)

If $(X, A)$ are jointly multivariate normal with all parameters unknown, then the least squares estimator $\gamma_{i}\hat{\xi }_{i}$ is the UMVU estimator of $\gamma_{i}\xi_{i}$ .

(b)

If the distribution of $A$ is unknown, then the least squares estimator $\gamma_{i}\hat{\xi }_{i}$ is UMVU among all linear estimators of $\gamma_{i}\xi_{i}$ .

(c)

If $\mathbb {E}\left[AA'\right]$ is known, no best linear unbiased estimator of $\gamma_{i}\xi_{i}$ exists.

If the distribution of $\mathbf{X}_{i}$ is completely unknown, show that, for any $j, j=1, \ldots , p$ , the $c_{j}$ -order statistics of $(X_{1}, \ldots , X_{n})$ are complete sufficient. (That is, the vectors $X_{1}, \ldots , X_{n}$ are ordered according to their $j$ th coordinate.)
Let $Y_{1 \times n}$ be a random variable with unknown distribution (possibly different from $\mathbf{X}_{i}$ ). Form the $(p-1) \times n$ matrix $\binom {\mathbf{x}}{\mathbf{y}}$ , and for any $j=1, \ldots , p$ , calculate the $c_{j}$ -order statistics based on the columns of $\binom {\mathbf{x}}{\mathbf{y}}$ . Show that these $c_{j}$ -order statistics are sufficient.
Use parts (a) and (b) to prove Theorem 4.14(b).

References

[LeCa98] Lehmann E.L., Casella G. Theory of Point Estimation. 2nd ed., Springer, 1998.