Math Problems Vault

Problem 1.8.1

(a)(b)

Lemma 9.3 states: a sufficient condition for $Y_{n}$ to converge in probability to $c$ is that $\mathbb {E}\left[(Y_{n}-c)^{2}\right] \rightarrow 0$ .

?

(a)

Prove Chebychev's Inequality: For any random variable $X$ and non-negative function $g(\cdot )$ ,

\mathbb {P}\left(g(X) \geq \varepsilon \right) \leq \frac{1}{\varepsilon } \mathbb {E}\left[g(X)\right]

for every $\varepsilon >0$ . (In many statistical applications, it is useful to take $g(x)= (x-a)^{2} / b^{2}$ for some constants $a$ and $b$ .)

(b)

Prove Lemma 9.3. [Hint: Apply Chebychev's Inequality.]

Problem 1.8.2

To see that the converse of Theorem 8.2 does not hold, let $X_{1}, \ldots , X_{n}$ be iid with $\mathbb {E}\left[X_{i}\right]=\theta , \operatorname {var}\left(X_{i}\right)=\sigma^{2}<\infty$ , and let $\delta_{n}=\bar{X}$ with probability $1-\varepsilon_{n}$ and $\delta_{n}=A_{n}$ with probability $\varepsilon_{n}$ . If $\varepsilon_{n}$ and $A_{n}$ are constants satisfying

\varepsilon _{n} \rightarrow 0 \quad \text{ and } \quad \varepsilon _{n} A_{n} \rightarrow \infty ,

then $\delta_{n}$ is consistent for estimating $\theta$ , but $\mathbb {E}\left[(\delta_{n}-\theta )^{2}\right]$ does not tend to zero.

?

Problem 1.8.3

Suppose $\rho (x)$ is an even function, nondecreasing and non-negative for $x \geq 0$ and positive for $x>0$ . Then, $\mathbb {E}\left[\rho \left[\delta_{n}-g(\theta )\right]\right] \rightarrow 0$ for all $\theta$ implies that $\delta_{n}$ is consistent for estimating $g(\theta )$ .

?

Problem 1.8.4

(a)(b)

?

(a)

If $A_{n}, B_{n}$ , and $Y_{n}$ tend in probability to $a, b$ , and $y$ , respectively, then $A_{n}+B_{n} Y_{n}$ tends in probability to $a+b y$ .

(b)

If $A_{n}$ takes on the constant value $a_{n}$ with probability 1 and $a_{n} \rightarrow a$ , then $A_{n} \rightarrow a$ in probability.

Problem 1.8.5

Referring to Example 8.4, show that $c_{n} S_{n}^{2} \xrightarrow {P} \sigma^{2}$ for any sequence of constants $c_{n} \rightarrow 1$ . In particular, the MLE $\hat{\sigma }^{2}=\frac{n-1}{n} S_{n}^{2}$ is a consistent estimator of $\sigma^{2}$ .

?

Problem 1.8.6

Verify Equation (8.9).

?

Problem 1.8.7

If $\left\{ a_{n}\right\}$ is a sequence of real numbers tending to $a$ , and if $b_{n}=\left(a_{1}+\ldots +a_{n}\right) / n$ , then $b_{n} \rightarrow a$ .

?

Problem 1.8.8

(a)(b)

?

(a)

If $\delta_{n}$ is consistent for $\theta$ , and $g$ is continuous, then $g\left(\delta_{n}\right)$ is consistent for $g(\theta )$ .

(b)

Let $X_{1}, \ldots , X_{n}$ be iid as $N(\theta , 1)$ , and let $g(\theta )=0$ if $\theta \neq 0$ and $g(0)=1$ . Find a consistent estimator of $g(\theta )$ .

Problem 1.8.9

(a)(b)

?

(a)

In Example 8.5, find $\operatorname {cov}\left(X_{i}, X_{j}\right)$ for any $i \neq j$ .

(b)

Verify (8.10).

Problem 1.8.10

(a)(b)

?

(a)

In Example 8.5, find the value of $p_{1}$ for which $p_{k}$ becomes independent of $k$ .

(b)

If $p_{1}$ has the value given in (a), then for any integers $i_{1}<\cdots <i_{r}$ and $k$ , the joint distribution of $X_{i_{1}}, \ldots , X_{i_{r}}$ is the same as that of $X_{i_{1}+k}, \ldots , X_{i_{r}+k}$ .

Problem 1.8.11

(a)(b)

Suppose $X_{1}, \ldots , X_{n}$ have a common mean $\xi$ and variance $\sigma^{2}$ , and that $\operatorname {cov}\left(X_{i}, X_{j}\right)= \rho_{j-i}$ . For estimating $\xi$ , show that:

?

(a)

$\bar{X}$ is not consistent if $\rho_{j-i}=\rho \neq 0$ for all $i \neq j$ ;

(b)

$\bar{X}$ is consistent if $\left|\rho_{j-i}\right| \leq M \gamma^{j-i}$ with $\left|\gamma \right|<1$ .

Problem 1.8.12

(a)(b)(c)

Suppose that $k_{n}\left[\delta_{n}-g(\theta )\right]$ tends in law to a continuous limit distribution $H$ . Prove that:

?

(a)

If $k_{n}^{\prime } / k_{n} \rightarrow d \neq 0$ or $\infty$ , then $k_{n}^{\prime }\left[\delta_{n}-g(\theta )\right]$ also tends to a continuous limit distribution.

(b)

If $k_{n}^{\prime } / k_{n} \rightarrow 0$ or $\infty$ , then $k_{n}^{\prime }\left[\delta_{n}-g(\theta )\right]$ tends in probability to zero or infinity, respectively.

(c)

If $k_{n} \rightarrow \infty$ , then $\delta_{n} \rightarrow g(\theta )$ in probability.

Problem 1.8.13

Show that if $Y_{n} \rightarrow c$ in probability, then it tends in law to a random variable $Y$ which is equal to $c$ with probability 1.

?

Problem 1.8.14

(a)(b)(c)

?

(a)

In Example 8.7(i) and (ii), $Y_{n} \rightarrow 0$ in probability. Show that:

(b)

If $H_{n}$ denotes the distribution function of $Y_{n}$ in Example 8.7(i) and (ii), then $H_{n}(a) \rightarrow 0$ for all $a<0$ and $H_{n}(a) \rightarrow 1$ for all $a>0$ .

(c)

Determine $\lim H_{n}(0)$ for Example 8.7(i) and (ii).

Problem 1.8.15

(a)(b)

If $T_{n}>0$ satisfies $\sqrt{n}\left[T_{n}-\theta \right] \xrightarrow {\mathcal{L}} N\left(0, \tau^{2}\right)$ , find the limiting distribution of (suitably normalized)

?

(a)

$\sqrt{T_{n}}$ and

(b)

$\log T_{n}$ .

Problem 1.8.16

(a)(b)(c)(d)

If $T_{n}$ satisfies $\sqrt{n}\left[T_{n}-\theta \right] \xrightarrow {\mathcal{L}} N\left(0, \tau^{2}\right)$ , find the limiting distribution of (suitably normalized)

?

(a)

$T_{n}^{2}$ ,

(b)

$\log \left|T_{n}\right|$ ,

(c)

$1 / T_{n}$ , and

(d)

$e^{T_{n}}$ .

Problem 1.8.17

(a)(b)

Variance stabilizing transformations are transformations for which the resulting statistic has an asymptotic variance that is independent of the parameters of interest. For each of the following cases, find the asymptotic distribution of the transformed statistic and show that it is variance stabilizing.

?

(a)

$T_{n}=\frac{1}{n} \sum_{i=1}^{n} X_{i}, X_{i} \sim \operatorname {Poisson}(\lambda ), h\left(T_{n}\right)=\sqrt{T_{n}}$ .

(b)

$T_{n}=\frac{1}{n} \sum_{i=1}^{n} X_{i}, X_{i} \sim \operatorname {Bernoulli}(p), h\left(T_{n}\right)=\arcsin \sqrt{T_{n}}$ .

Problem 1.8.18

(a)(b)

?

(a)

The function $v(\cdot )$ is a variance stabilizing transformation if the estimator $v\left(T_{n}\right)$ has asymptotic variance $\tau^{2}(\theta )\left[v^{\prime }(\theta )\right]^{2}=c$ , where $c$ is a constant independent of $\theta$ .

(b)

For any positive integer $n$ , find the variance stabilizing transformation if $\tau^{2}(\theta )=\theta^{n}$ . In particular, be careful of the important case $n=2$ .

[A variance stabilizing transformation (if it exists) is the solution of a differential equation resulting from the Delta Method approximation of the variance of an estimator (Theorem 8.12) and is not a function of the distribution of the statistic (other than the fact that the distribution will determine the form of the variance). The transformations of part (b) are known as the Box-Cox family of power transformations and play an important role in applied statistics. For more details and interesting discussions, see Bickel and Doksum 1981, Box and Cox 1982, and Hinkley and Runger 1984.]

Problem 1.8.19

(a)(b)

Serfling (1980, Section 3.1) remarks that the following variations of Theorem 8.12 can be established. Show that:

?

(a)

If $h$ is differentiable in a neighborhood of $\theta$ , and $h^{\prime }$ is continuous at $\theta$ , then $h^{\prime }(\theta )$ may be replace by $h^{\prime }\left(T_{n}\right)$ to obtain

\sqrt{n} \frac{\left[h\left(T_{n}\right)-h(\theta )\right]}{\tau h^{\prime }\left(T_{n}\right)} \xrightarrow {\mathcal{L}} N(0,1).

(b)

Furthermore, if $\tau^{2}$ is a continuous function of $\theta$ , say $\tau^{2}(\theta )$ , it can be replaced by $\tau^{2}\left(T_{n}\right)$ to obtain

\sqrt{n} \frac{\left[h\left(T_{n}\right)-h(\theta )\right]}{\tau \left(T_{n}\right) h^{\prime }\left(T_{n}\right)} \xrightarrow {\mathcal{L}} N(0,1).

Problem 1.8.20

Theorem 8.16 states (it starts in a multivariate setting, but its conclusion is univariate): let $(X_{1 \nu }, \ldots , X_{s \nu })$ , $\nu =1, \ldots , n$ , be $n$ independent $s$ -tuples of random variables with $\mathbb {E}\left[X_{i \nu }\right]=\xi_{i}$ and $\operatorname {cov}(X_{i \nu }, X_{j \nu })=\sigma_{i j}$ . Let $\bar{X}_{i}=\sum_{\nu } X_{i \nu } / n$ , and suppose that $h$ is a real-valued function of $s$ arguments with continuous first partial derivatives. Then,

\sqrt{n}\left[h\left(\bar{X}_{1}, \ldots , \bar{X}_{s}\right)-h\left(\xi _{1}, \ldots , \xi _{s}\right)\right] \xrightarrow {\mathcal{L}} N\left(0, v^{2}\right),

where $v^{2}=\sum_{i,j} \sigma_{i j} \dfrac {\partial h}{\partial \xi_{i}} \dfrac {\partial h}{\partial \xi_{j}}$ , provided $v^{2}>0$ .

Prove Theorem 8.16.

?

Problem 1.8.21

(a)(b)(c)(d)

A sequence of numbers $R_{n}$ is said to be $o\left(1 / k_{n}\right)$ as $n \rightarrow \infty$ if $k_{n} R_{n} \rightarrow 0$ and to be $O\left(1 / k_{n}\right)$ if there exist $M$ and $n_{0}$ such that $\left|k_{n} R_{n}\right|<M$ for all $n>n_{0}$ or, equivalently, if $k_{n} R_{n}$ is bounded.

?

(a)

If $R_{n}=o\left(1 / k_{n}\right)$ , then $R_{n}=0\left(1 / k_{n}\right)$ .

(b)

$R_{n}=0(1)$ if and only if $R_{n}$ is bounded.

(c)

$R_{n}=o(1)$ if and only if $R_{n} \rightarrow 0$ .

(d)

If $R_{n}$ is $O\left(1 / k_{n}\right)$ and $k_{n}^{\prime } / k_{n}$ tends to a finite limit, then $R_{n}$ is $O\left(1 / k_{n}^{\prime }\right)$ .

Problem 1.8.22

(a)(b)

?

(a)

If $R_{n}$ and $R_{n}^{\prime }$ are both $O\left(1 / k_{n}\right)$ , so is $R_{n}+R_{n}^{\prime }$ .

(b)

If $R_{n}$ and $R_{n}^{\prime }$ are both $o\left(1 / k_{n}\right)$ , so is $R_{n}+R_{n}^{\prime }$ .

Problem 1.8.23

(a)(b)

Suppose $k_{n}^{\prime } / k_{n} \rightarrow \infty$ .

?

(a)

If $R_{n}=0\left(1 / k_{n}\right)$ and $R_{n}^{\prime }=0\left(1 / k_{n}^{\prime }\right)$ , then $R_{n}+R_{n}^{\prime }=0\left(1 / k_{n}\right)$ .

(b)

If $R_{n}=o\left(1 / k_{n}\right)$ and $R_{n}^{\prime }=o\left(1 / k_{n}^{\prime }\right)$ , then $R_{n}+R_{n}^{\prime }=o\left(1 / k_{n}\right)$ .

Problem 1.8.24

A sequence of random variables $Y_{n}$ is bounded in probability if given any $\varepsilon >0$ , there exist $M$ and $n_{0}$ such that $\mathbb {P}\left(\left|Y_{n}\right|>M\right)<\varepsilon$ for all $n>n_{0}$ . Show that if $Y_{n}$ converges in law, then $Y_{n}$ is bounded in probability.

?

Problem 1.8.25

In generalization of the notation $o$ and $O$ , let us say that $Y_{n}=o_{p}\left(1 / k_{n}\right)$ if $k_{n} Y_{n} \rightarrow 0$ in probability and that $Y_{n}=O_{p}\left(1 / k_{n}\right)$ if $k_{n} Y_{n}$ is bounded in probability. Show that the results of Problems 8.21-8.23 continue to hold if $o$ and $O$ are replaced by $o_{p}$ and $O_{p}$ .

?

Problem 1.8.26

(a)(b)

Let $(X_{n}, Y_{n})$ have a bivariate normal distribution with means $\mathbb {E}\left[X_{n}\right]=\mathbb {E}\left[Y_{n}\right]=0$ , variances $\mathbb {E}\left[X_{n}^{2}\right]=\mathbb {E}\left[Y_{n}^{2}\right]=1$ , and with correlation coefficient $\rho_{n}$ tending to 1 as $n \rightarrow \infty$ .

?

(a)

Show that $\left(X_{n}, Y_{n}\right) \xrightarrow {\mathcal{L}}(X, Y)$ where $X$ is $N(0,1)$ and $\mathbb {P}\left(X=Y\right)=1$ .

(b)

If $S=\left\{ (x, y): x=y\right\}$ , show that (8.25) does not hold.

Problem 1.8.27

Theorem 8.22 states: suppose that $[\sqrt{n}(Y_{1n}-\theta_{1}), \ldots , \sqrt{n}(Y_{rn}-\theta_{r})]$ tends in law to the multivariate normal distribution with mean vector $0$ and covariance matrix $\Sigma$ , and suppose that $h_{1}, \ldots , h_{r}$ are $r$ real-valued functions of $\theta =(\theta_{1}, \ldots , \theta_{r})$ , defined and continuously differentiable in a neighborhood $\omega$ of the parameter point $\theta$ and such that the matrix $B=\left\| \partial h_{i} / \partial \theta_{j}\right\|$ of partial derivatives is nonsingular in $\omega$ . Then, $[\sqrt{n}[h_{1}(Y_{n})-h_{1}(\theta )], \ldots , \sqrt{n}[h_{r}(Y_{n})-h_{r}(\theta )]]$ tends in law to the multivariate normal distribution with mean vector $0$ and covariance matrix $B\Sigma B'$ .

Prove Theorem 8.22.

?