Performance Evaluations in Large Samples

Let $X_{1}, \ldots , X_{n}$ be iid with $\mathbb {E}\left[X_{i}\right]=\xi$.

For fixed $n$, describe the relative error in Example 1.3 as a function of $p$.

Theorem 1.1 states: let $X_1, \ldots , X_n$ be iid with $\mathbb {E}\left[X_1\right]=\xi$, $\operatorname {Var}\left[X_1\right]=\sigma^2$, and finite fourth moment, and suppose $h$ is a function of a real variable whose first four derivatives $h', h'', h''', h^{(iv)}$ exist for all $x \in I$, where $I$ is an interval with $\mathbb {P}\left(X_1 \in I\right)=1$. Suppose further that $\left|h^{(iv)}(x)\right| \leq M$ for all $x \in I$, for some $M<\infty$. Then,

and if, in addition, the fourth derivative of $h^2$ is also bounded,

where the remainder $R_n$ in both cases is $O(1/n^2)$.

Theorem 1.5 states: suppose that the assumptions of Theorem 1.1 hold, and that $c_n$ is a sequence of constants satisfying $c_{n}=1+\dfrac {a}{n}+O\! \left(\dfrac {1}{n^{2}}\right)$. Then, the variance of $\delta_{n}(\bar{X})=h\left(c_{n} \bar{X}\right)$ satisfies

Prove Theorem 1.5.

Let $X_{1}, \ldots , X_{n}$ be iid as $N\left(\xi , \sigma^{2}\right), \sigma^{2}$ known, and let $g(\xi )=\xi^{r}, r=2,3,4$. Determine, up to terms of order $1 / n$,

Let $X_{1}, \ldots , X_{n}$ be iid as $N\left(\xi , \sigma^{2}\right), \xi$ known. For even $r$, determine the variance of the UMVU estimator (2.2.4) of $\sigma^{r}$ up to terms of order $r$.

Solve the preceding problem for the case that $\xi$ is unknown.

For estimating $p^{m}$ in Example 3.3.1, determine, up to order $1 / n$,

Solve the preceding problem if $p^{m}$ is replaced by the estimand of Problem 2.3.3.

Let $X_{1}, \ldots , X_{n}$ be iid as Poisson $P(\theta )$.

Solve part (b) of the preceding problem for the estimator (2.3.22).

Under the assumptions of Problem 1.1, show that $\mathbb {E}\left[\left|\bar{X}-\xi \right|^{2 k-1}\right]=O\left(n^{-k+1 / 2}\right)$.

Obtain a variant of Theorem 1.1, which requires existence and boundedness of only $h^{\prime \prime \prime }$ instead of $h^{(i v)}$, but where $R_{n}$ is only $O\left(n^{-3 / 2}\right)$.

To see that Theorem 1.1 is not necessarily valid without boundedness of the fourth (or some higher) derivative, suppose that the $X$ 's are distributed as $N\left(\xi , \sigma^{2}\right)$ and let $h(X)=e^{x^{4}}$. Then, all moments of the $X$ 's and all derivatives of $h$ exist.

Let $X_{1}, \ldots , X_{n}$ be iid from the exponential distribution with density $(1 / \theta ) e^{-x / \theta }, x>0$, and $\theta >0$.

Let $X_{1}, \ldots , X_{n}$ be iid according to $U(0, \theta )$. Determine the variance of the UMVU estimator of $\theta^{k}$, where $k$ is an integer, $k>-n$.

Under the assumptions of Problem 1.15, find the MLE of $\theta^{k}$ and compare its expected squared error with the variance of the UMVU estimator.

Let $X_{1}, \ldots , X_{n}$ be iid according to $U(0, \theta )$, let $T=\max \left(X_{1}, \ldots , X_{n}\right)$, and let $h$ be a function satisfying the conditions of Theorem 1.1. Show that

and

Apply the results of Problem 1.17 to obtain approximate answers to Problems 1.15 and 1.16, and compare the answers with the exact solutions.

If the $X$ 's are as in Theorem 1.1 and if the first five derivatives of $h$ exist and the fifth derivative is bounded, show that

and if the fifth derivative of $h^{2}$ is also bounded

where $\mu_{3}=\mathbb {E}\left[(X-\xi )^{3}\right]$.

Under the assumptions of the preceding problem, carry the calculation of the variance (1.16) to terms of order $1 / n^{2}$, and compare the result with that of the preceding problem.

Carry the calculation of Problem 1.4 to terms of order $1 / n^{2}$.

For the estimands of Problem 1.4, calculate the expected squared error of the MLE to terms of order $1 / n^{2}$, and compare it with the variance calculated in Problem 1.21.

Calculate the variance (1.18) to terms of order $1 / n^{2}$ and compare it with the expected squared error of the MLE carried to the same order.

Find the variance of the estimator (2.3.17) up to terms of the order $1 / n^{3}$.

For the situation of Example 1.12, show that the UMVU estimator $\delta_{1 n}$ is the biascorrected MLE, where the MLE is $\delta_{3 n}$.

For the estimators of Example 1.13:

On the basis of a sample from $N\left(\xi , \sigma^{2}\right)$, let $P_{n}(\xi , \sigma )$ be the probability that the UMVU estimator $\bar{X}^{2}-\sigma^{2} / n$ of $\xi^{2}$ ( $\sigma$ known) is negative.

Use the $t$-distribution to find the value of $P_{n}(0, \sigma )$ in the preceding problem for the UMVU estimator of $\xi^{2}$ when $\sigma$ is unknown for representative values of $n$.

Theorem 1.9 states: if a sequence of random variables $T_{n}$ tends to $\xi$ in probability and if $h$ is continuous at $\xi$, then $h(T_{n}) \xrightarrow {P} h(\xi )$. To show that $\mathbb {P}\left(\left|h(T_{n})-h(\xi )\right|<a\right) \rightarrow 1$, it is enough to notice that by the continuity of $h$, the difference $\left|h(T_{n})-h(\xi )\right|$ will be arbitrarily small if $T_{n}$ is close to $\xi$, and that for any $a$, the probability that $\left|T_{n}-\xi \right|<a$ tends to $1$ as $n \rightarrow \infty$.

Fill in the details of the proof of Theorem 1.9. (See also Problem 1.8.8.)

In Example 8.13 with $\theta =0$, show that $\delta_{2 n}$ is not exactly distributed as $\sigma^{2}\left(\chi_{1}^{2}-1\right) / n$.

In Example 8.13, let $\delta_{4 n}=\max \left(0, \bar{X}^{2}-\sigma^{2} / n\right)$, which is an improvement over $\delta_{1 n}$.

Let $X$ have the binomial distribution $b(p, n)$, and let $g(p)=p q$. The UMVU estimator of $g(p)$ is $\delta =X(n-X) / n(n-1)$. Determine the limit distribution of $\sqrt{n}(\delta - p q)$ and $n(\delta -p q)$ when $g^{\prime }(p) \neq 0$ and $g^{\prime }(p)=0$, respectively.

Let $X_{1}, \ldots , X_{n}$ be iid as $N(\xi , 1)$. Determine the limit behavior of the distribution of the UMVU estimator of $p=\mathbb {P}\left(\left|X_{i}\right| \leq u\right)$.

Determine the limit behavior of the estimator (2.3.22) as $n \rightarrow \infty$.

Let $X_{1}, \ldots , X_{n}$ be iid with distribution $P_{\theta }$, and suppose $\delta_{n}$ is UMVU for estimating $g(\theta )$ on the basis of $X_{1}, \ldots , X_{n}$. If there exists $n_{0}$ and an unbiased estimator $\delta_{0}\left(X_{1}, \ldots , X_{n_{0}}\right)$ which has finite variance for all $\theta$, then $\delta_{n}$ is consistent for $g(\theta )$.

Let $Y_{n}$ be distributed as $N(0,1)$ with probability $\pi_{n}$ and as $N\left(0, \tau_{n}^{2}\right)$ with probability $1-\pi_{n}$. If $\tau_{n} \rightarrow \infty$ and $\pi_{n} \rightarrow \pi$, determine for what values of $\pi$ the sequence $\left\{ Y_{n}\right\}$ does and does not have a limit distribution.

Let $b_{m, n}, m, n=1,2, \ldots$, be a double sequence of real numbers, which for each fixed $m$ is nondecreasing in $n$. Show that $\lim_{n \rightarrow \infty } \lim_{m \rightarrow \infty } b_{m, n}=\lim_{m, n \rightarrow \infty } \inf b_{m, n}$ and $\lim_{m \rightarrow \infty } \lim_{n \rightarrow \infty } b_{m, n}=\lim_{m, n \rightarrow \infty } \sup b_{m, n}$ provided the indicated limits exist (they may be infinite) and where $\lim \inf b_{m, n}$ and $\lim \sup b_{m, n}$ denote, respectively, the smallest and the largest limit points attainable by a sequence $b_{m_{k}, n_{k}}, k=1,2, \ldots$, with $m_{k} \rightarrow \infty$ and $n_{k} \rightarrow \infty$.