Math Problems Vault

Problem 5.1

Let $p\geq 1$ and $\mathcal F_p$ be the set of cumulative distribution functions on $\mathbb {R}$ having finite $p$ th moment. Mallows' distance between $F$ and $G$ in $\mathcal F_p$ is defined to be

\varrho _{M_p}(F,G)=\inf \left(\mathbb {E}\left[\left\| X-Y\right\| ^p\right]\right)^{1/p},

where the infimum is taken over all pairs of random variables $X$ and $Y$ having marginal distributions $F$ and $G$ , respectively. Show that $\varrho_{M_p}$ is a distance on $\mathcal F_p$ .

?

Remark.

?

Corresponding exercise in Shao (2003): 5.3.

Problem 5.2

(i)(ii)

Let $\mathcal F_1$ be the collection of cumulative distribution functions on $\mathbb {R}$ with finite means and $\varrho_{M_1}$ be as defined in Exercise 5.1. Show that

?

(i)

$\varrho_{M_1}(F,G)=\int_0^1\left|F^{-1}(t)-G^{-1}(t)\right|\, dt$ ;

(ii)

$\varrho_{M_1}(F,G)=\int_{-\infty }^{\infty }\left|F(x)-G(x)\right|\, dx$ .

Remark.

?

Recall that $\mathcal F_p$ is the collection of cumulative distribution functions on $\mathbb {R}$ with finite $p$ th moment, and $\varrho_{M_p}(F,G)=\inf \left[\mathbb {E}\left[\left|X-Y\right|^p\right]\right]^{1/p}$ , where the infimum ranges over joint distributions with marginals $F$ and $G$ with $p=1$ .

Corresponding exercise in Shao (2003): 5.5.

Problem 5.3

Let $\varrho_{M_p}$ be the Mallows' distance defined in Exercise 5.1 and $\left\{ G,G_1,G_2,\ldots \right\} \subset \mathcal F_p$ . Show that $\lim_n\varrho_{M_p}(G_n,G)=0$ if and only if

\lim _n\int \left|x\right|^p\, dG_n(x)=\int \left|x\right|^p\, dG(x)

and $\lim_nG_n(x)=G(x)$ for any $x$ at which $G$ is continuous.

?

Remark.

?

Recall that $\mathcal F_p$ is the collection of cumulative distribution functions on $\mathbb {R}$ with finite $p$ th moment, and $\varrho_{M_p}(F,G)=\inf \left[\mathbb {E}\left[\left|X-Y\right|^p\right]\right]^{1/p}$ , where the infimum ranges over joint distributions with marginals $F$ and $G$ ( $p\geq 1$ ).

Problem 5.4

(i)(ii)

Find an example of cumulative distribution functions $G,G_1,G_2,\ldots$ on $\mathbb {R}$ such that

?

(i)

$\lim_n\varrho_\infty (G_n,G)=0$ but $\varrho_{M_p}(G_n,G)$ does not converge to 0, where $\varrho_{M_p}$ is the distance defined in Exercise 5.1 and $\varrho_\infty$ is the sup-norm distance defined as $\varrho_\infty (F,G)=\sup_x\left|F(x)-G(x)\right|$ for any cumulative distribution functions $F$ and $G$ ;

(ii)

$\lim_n\varrho_{M_p}(G_n,G)=0$ but $\varrho_\infty (G_n,G)$ does not converge to 0.

Remark.

?

Recall that $\mathcal F_p$ is the collection of cumulative distribution functions on $\mathbb {R}$ with finite $p$ th moment, and $\varrho_{M_p}(F,G)=\inf \left[\mathbb {E}\left[\left|X-Y\right|^p\right]\right]^{1/p}$ , where the infimum ranges over joint distributions with marginals $F$ and $G$ ( $p\geq 1$ ).

Corresponding exercise in Shao (2003): 5.6.

Problem 5.5

(i)(ii)

Let $X$ be a random variable having cumulative distribution function $F$ . Show that

?

(i)

$\mathbb {E}\left[\left|X\right|^2\right]<\infty$ implies $\int \left\{ F(t)[1-F(t)]\right\}^{p/2}\, dt<\infty$ for $p>1$ ;

(ii)

$\mathbb {E}\left[\left|X\right|^{2+\delta }\right]<\infty$ with some $\delta >0$ implies $\int \left\{ F(t)[1-F(t)]\right\}^{1/2}\, dt<\infty$ .

Remark.

?

Corresponding exercise in Shao (2003): 5.8.

Problem 5.6

Show that $p_i=c/n$ , $i=1,\ldots ,n$ , $\lambda =-(c/n)^{n-1}$ is a maximum of the function

H(p_1,\ldots ,p_n,\lambda )=\prod _{i=1}^n p_i+\lambda \left(\sum _{i=1}^n p_i-c\right)

over $p_i>0$ , $i=1,\ldots ,n$ , $\sum_{i=1}^n p_i=c$ .

?

Remark.

?

This exercise shows that the empirical distribution function (which puts mass $n^{-1}$ to each of $n$ observations) is a maximum likelihood estimator.

Corresponding exercise in Shao (2003): 5.10.

Problem 5.7

Consider the problem of maximizing $\prod_{i=1}^n p_i$ over

p_i>0,\quad i=1,\ldots ,n,\quad \sum _{i=1}^n p_i=1,\quad \text{and}\quad \sum _{i=1}^n p_i u_i=0,

where $u_i$ 's are $s$ -vectors. Show that the solution is

\widehat{p}_i=\frac1{n(1+\lambda ^\tau u_i)},\quad i=1,\ldots ,n,

and $\lambda \in \mathbb {R}^{s}$ satisfying

\sum _{i=1}^n\frac{u_i}{1+\lambda ^\tau u_i}=0.

?

Remark.

?

Corresponding exercise in Shao (2003): 5.11.

Problem 5.8

(i)(ii)(iii)(iv)

Let $\delta_1,\ldots ,\delta_n$ be $n$ observations from a binary random variable and

\ell (p_1,\ldots ,p_{n+1})=\prod _{i=1}^n p_i^{\delta _i}\left(\sum _{j=i+1}^{n+1}p_j\right)^{1-\delta _i}.

?

(i)

Show that maximizing $\ell (p_1,\ldots ,p_{n+1})$ subject to $p_i\geq 0$ , $i=1,\ldots ,n+1$ , $\sum_{i=1}^{n+1}p_i=1$ is equivalent to maximizing

\prod _{i=1}^n q_i^{\delta _i}(1-q_i)^{n-i+1-\delta _i},

where $q_i=p_i/\sum_{j=i}^{n+1}p_j$ , $i=1,\ldots ,n$ .

(ii)

Show that

\widehat{p}_i=\frac{\delta _i}{n-i+1}\prod _{j=1}^{i-1}\left(1-\frac{\delta _j}{n-j+1}\right),\quad i=1,\ldots ,n,\quad \widehat{p}_{n+1}=1-\sum _{i=1}^n\widehat{p}_i

maximizes $\ell (p_1,\ldots ,p_{n+1})$ subject to $p_i\geq 0$ , $i=1,\ldots ,n+1$ , $\sum_{i=1}^{n+1}p_i=1$ .

(iii)

For any $x_1\leq x_2\leq \cdots \leq x_n$ , show that

\widehat{F}(t)=\sum _{i=1}^{n+1}\widehat{p}_i\; \mathbb {1}_{(0,t]}(x_i)=1-\prod _{x_i\leq t}\left(1-\frac{\delta _i}{n-i+1}\right).

(iv)

When $\delta_i=1$ for all $i$ , show that $\widehat{p}_i=n^{-1}$ , $i=1,\ldots ,n$ , and $\widehat{p}_{n+1}=0$ .

Remark.

?

This exercise shows that the well-known Kaplan--Meier product-limit estimator is a maximum likelihood estimator.

Corresponding exercise in Shao (2003): 5.15.

Problem 5.9

(i)(ii)(iii)(iv)

Let $(X_1,\ldots ,X_n)$ be a random sample of random variables with Lebesgue density $f$ ,

F_n(t)=\frac1n\sum _{i=1}^n\; \mathbb {1}_{(-\infty ,t]}(X_i),\quad t\in \mathbb {R},

be the empirical distribution, and

f_n(t)=\frac{F_n(t+\lambda _n)-F_n(t-\lambda _n)}{2\lambda _n},\quad t\in \mathbb {R},

where $\left\{ \lambda_n\right\}$ is a sequence of positive constants.

?

(i)

Show that $f_n$ is a Lebesgue density on $\mathbb {R}$ .

(ii)

Suppose that $f$ is continuously differentiable at $t$ , $\lim_n\lambda_n=0$ , and $\lim_n n\lambda_n=\infty$ . Show that the mean squared error of $f_n(t)$ as an estimator of $f(t)$ equals

\frac{f(t)}{2n\lambda _n}+o\left(\frac1{n\lambda _n}\right)+O(\lambda _n^2)

as $n\to \infty$ .

(iii)

Under $\lim_n n\lambda_n^3=0$ and the conditions in (ii), show that

\sqrt{n\lambda _n}[f_n(t)-f(t)]\to _d\mathscr {N}\left(0, f(t)/2\right).

(iv)

Suppose that $f$ is continuous on $[a,b]$ , $-\infty <a<b<\infty$ , $\lim_n\lambda_n=0$ , and $\lim_n n\lambda_n=\infty$ . Show that

\int _a^b f_n(t)\, dt\to _p\int _a^b f(t)\, dt.

Remark.

?

Corresponding exercise in Shao (2003): 5.16.

Problem 5.10

(i)(ii)(iii)(iv)

Let $(X_1,\ldots ,X_n)$ be a random sample of random variables with Lebesgue density $f$ , $w$ be a known Lebesgue density on $\mathbb {R}$ , and

\widehat{f}(t)=\frac1{n\lambda _n}\sum _{i=1}^n w\left(\frac{t-X_i}{\lambda _n}\right),

where $\left\{ \lambda_n\right\}$ is a sequence of positive constants.

?

(i)

Show that $\widehat{f}$ is a Lebesgue density on $\mathbb {R}$ .

(ii)

Show that if $\lambda_n\to 0$ , $n\lambda_n\to \infty$ , and $f$ is bounded and continuous at $t$ and $w_0=\int_{-\infty }^\infty [w(t)]^2\, dt<\infty$ , then

\sqrt{n\lambda _n}\left\{ \widehat{f}(t)-\mathbb {E}\left[\widehat{f}(t)\right]\right\} \to _d\mathscr {N}\left(0, w_0f(t)\right).

(iii)

Assume that $\lim_n n\lambda_n^3=0$ , $\int_{-\infty }^\infty \left|t\right|w(t)\, dt<\infty$ , $f'$ is bounded and continuous at $t$ , and the conditions in (ii) hold. Show that

\sqrt{n\lambda _n}[\widehat{f}(t)-f(t)]\to _d\mathscr {N}\left(0, w_0f(t)\right).

(iv)

Suppose that $\lambda_n\to 0$ , $n\lambda_n\to \infty$ , $w$ is bounded, and $f$ is bounded and continuous on $[a,b]$ , $-\infty <a<b<\infty$ . Show that $\int_a^b\widehat{f}(t)\, dt\to_p\int_a^b f(t)\, dt$ .

Remark.

?

Corresponding exercise in Shao (2003): 5.17.

Problem 5.11

Let $\ell (\theta ,\xi )$ be a likelihood function. Show that a maximum profile likelihood estimator $\widehat{\theta }$ of $\theta$ is an MLE if $\xi (\theta )$ , the maximum of $\sup_\xi \ell (\theta ,\xi )$ for a fixed $\theta$ , does not depend on $\theta$ .

?

Remark.

?

A maximum profile likelihood estimator $\widehat{\theta }$ maximizes the profile likelihood function $\ell_P(\theta )=\ell (\theta ,\xi (\theta ))$ , where $\ell (\theta ,\xi (\theta ))=\sup_\xi \ell (\theta ,\xi )$ for each fixed $\theta$ .

Corresponding exercise in Shao (2003): 5.20.

Problem 5.12

Let $(X_1,\ldots ,X_n)$ be a random sample from $\mathscr {N}\left(\mu , \sigma^2\right)$ . Derive the profile likelihood function for $\mu$ or $\sigma^2$ . Discuss in each case whether the maximum profile likelihood estimator is the same as the MLE.

?

Remark.

?

Corresponding exercise in Shao (2003): 5.21.

Problem 5.13

(i)(ii)(iii)

Let $(X_1,\ldots ,X_n)$ be a random sample from a distribution $F$ and let $\pi (x)=\mathbb {P}\left( \delta_i=1 \mid X_i=x \right)$ , where $\delta_i=1$ if $X_i$ is observed and $\delta_i=0$ if $X_i$ is missing. Assume that $0<\pi =\int \pi (x)\, dF(x)<1$ .

?

(i)

Let $F_1(x)=\mathbb {P}\left( X_i\leq x \mid \delta_i=1 \right)$ . Show that $F$ and $F_1$ are the same if and only if $\pi (x)\equiv \pi$ .

(ii)

Let $\widehat{F}$ be the empirical distribution putting mass $r^{-1}$ to each observed $X_i$ , where $r$ is the number of observed $X_i$ 's. Show that $\widehat{F}(x)$ is unbiased and consistent for $F_1(x)$ , $x\in \mathbb {R}$ .

(iii)

When $\pi (x)\equiv \pi$ , show that $\widehat{F}(x)$ in part (ii) is unbiased and consistent for $F(x)$ , $x\in \mathbb {R}$ . When $\pi (x)$ is not constant, show that $\widehat{F}(x)$ is biased and inconsistent for $F(x)$ for some $x\in \mathbb {R}$ .

Remark.

?

Corresponding exercise in Shao (2003): 5.23.

Problem 5.14

Let $\mathcal F$ be a collection of distributions on $\mathbb {R}^{d}$ . A functional $T$ defined on $\mathcal F$ is Gateaux differentiable at $G\in \mathcal F$ if and only if there is a linear functional $L_G$ on $\mathcal D=\left\{ c(G_1-G_2):c\in \mathbb {R},G_j\in \mathcal F,j=1,2\right\}$ (i.e., $L_G(c_1\Delta_1+c_2\Delta_2)=c_1L_G(\Delta_1)+c_2L_G(\Delta_2)$ for any $\Delta_j\in \mathcal D$ and $c_j\in \mathbb {R}$ ) such that $\Delta \in \mathcal D$ and $G+t\Delta \in \mathcal F$ imply

\lim _{t\to 0}\left[\frac{T(G+t\Delta )-T(G)}t-L_G(\Delta )\right]=0.

Assume that the functional $L_F$ is continuous in the sense that $\left\| \Delta_j-\Delta \right\|_{\infty }\to 0$ implies $L_F(\Delta_j)\to L_F(\Delta )$ , where $D\in \mathcal D$ , $D_j\in \mathcal D$ , and $\left\| D\right\|_{\infty }=\sup_x\left|D(x)\right|$ for any $D\in \mathcal D$ is the sup-norm. Show that $\phi_F(x)=L_F(\delta_x-F)$ is a bounded function of $x$ , where $\delta_x$ is the degenerated distribution at $x$ .

?

Remark.

?

Corresponding exercise in Shao (2003): 5.25.

Problem 5.15

Suppose that a functional $T$ is Gateaux differentiable at $F$ with a bounded and continuous influence function $\phi_F(x)=L_F(\delta_x-F)$ , where $\delta_x$ is the degenerated distribution at $x$ . Show that $L_F$ is continuous in the sense described in the previous exercise.

?

Remark.

?

Here $\mathcal D=\left\{ c(G_1-G_2):c\in \mathbb {R},\ G_1,G_2\in \mathcal F\right\}$ and $L_F$ is the linear differential satisfying $\lim_{t\to 0}[T(F+t\Delta )-T(F)]/t=L_F(\Delta )$ for admissible $\Delta \in \mathcal D$ . Continuity means $L_F(D_j)\to L_F(D)$ whenever $D_j,D\in \mathcal D$ and $\sup_x\left|D_j(x)-D(x)\right|\to 0$ .

Corresponding exercise in Shao (2003): 5.26.

Problem 5.16

Let $\mathcal F$ be the collection of all distributions on $\mathbb {R}$ and $z$ be a fixed real number. Define

T(G)=\int G(z-y)\, dG(y),\quad G\in \mathcal F.

Obtain the influence function $\phi_F$ for $T$ and show that $\phi_F$ is continuous if and only if $F$ is continuous.

?

Remark.

?

Corresponding exercise in Shao (2003): 5.29.

Problem 5.17

(i)(ii)

An L-functional is defined as

T(G)=\int xJ(G(x))\, dG(x),\quad G\in \mathcal F_0,

where $\mathcal F_0$ contains all distributions on $\mathbb {R}$ for which $T$ is well defined and $J(t)$ is a Borel function on $[0,1]$ .

?

(i)

Show that the influence function is

\phi _F(x)=-\int _{-\infty }^\infty (\delta _x-F)(y)J(F(y))\, dy,

where $\delta_x$ is the degenerated distribution at $x$ .

(ii)

Show that $\int \phi_F(x)\, dF(x)=0$ and, if $J$ is bounded and $F$ has a finite second moment, then $\int [\phi_F(x)]^2\, dF(x)<\infty$ .

Remark.

?

Corresponding exercise in Shao (2003): 5.34.

Problem 5.18

(i)(ii)(iii)

Obtain explicit forms of the influence functions for L-functionals (Exercise 5.17) in the following cases and discuss which of them are bounded and continuous.

?

(i)

$J\equiv 1$ .

(ii)

$J(t)=4t-2$ .

(iii)

$J(t)=(\beta -\alpha )^{-1}\; \mathbb {1}_{(\alpha ,\beta )}(t)$ for some constants $\alpha <\beta$ .

Remark.

?

An L-functional is $T(G)=\int xJ(G(x))\, dG(x)$ on the distributions for which the integral is well defined, with $J$ a Borel function on $[0,1]$ . Its influence function is $\phi_F(x)=L_F(\delta_x-F)$ , where $L_F$ is the Gateaux differential and $\delta_x$ is the point mass at $x$ .

Corresponding exercise in Shao (2003): 5.37.

Problem 5.19

In part (iii) of the previous exercise, show that if $F$ is continuous at $F^{-1}(\alpha )$ and $F^{-1}(\beta )$ , then

\phi _F(x)=\begin{cases} \displaystyle \frac{F^{-1}(\alpha )(1-\alpha )-F^{-1}(\beta )(1-\beta )}{\beta -\alpha }-T(F)& x<F^{-1}(\alpha ),\\ ]6pt] \displaystyle \frac{x-F^{-1}(\alpha )\alpha -F^{-1}(\beta )(1-\beta )}{\beta -\alpha }-T(F)& F^{-1}(\alpha )\leq x\leq F^{-1}(\beta ),\\[6pt] \displaystyle \frac{F^{-1}(\beta )\beta -F^{-1}(\alpha )\alpha }{\beta -\alpha }-T(F)& x>F^{-1}(\beta ). \end{cases}

?

Remark.

?

An L-functional is $T(G)=\int xJ(G(x))\, dG(x)$ on the distributions for which the integral is well defined, with $J$ a Borel function on $[0,1]$ . Its influence function is $\phi_F(x)=L_F(\delta_x-F)$ , where $L_F$ is the Gateaux differential and $\delta_x$ is the point mass at $x$ . Here $J(t)=(\beta -\alpha )^{-1}\; \mathbb {1}_{(\alpha ,\beta )}(t)$ , as in Exercise 5.18(iii).

Problem 5.20

(i)(ii)(iii)

Let $T$ be an L-functional defined in Exercise 5.17.

?

(i)

Show that $T(F)=\theta$ if $F$ is symmetric about $\theta$ , $J$ is symmetric about $\frac12$ , and $\int_0^1J(t)\, dt=1$ .

(ii)

Assume that

\sigma _F^2=\int _{-\infty }^\infty \int _{-\infty }^\infty J(F(x))J(F(y))[F(\min \left\{ x,y\right\} )-F(x)F(y)]\, dx\, dy

is finite. Show that $\sigma^2=\int [\phi_F(x)]^2\, dF(x)$ , where $\phi_F$ is the influence function of $T$ .

(iii)

Show that if $J\equiv 1$ , then $\sigma_F^2$ in (ii) is equal to the variance of $F$ .

Remark.

?

An L-functional is $T(G)=\int xJ(G(x))\, dG(x)$ on the distributions for which the integral is well defined, with $J$ a Borel function on $[0,1]$ . Its influence function is $\phi_F(x)=L_F(\delta_x-F)$ , where $L_F$ is the Gateaux differential and $\delta_x$ is the point mass at $x$ .

Corresponding exercises in Shao (2003): 5.67, 5.69, 5.74.

Problem 5.21

(i)(ii)(iii)(iv)

Let $T$ be an L-functional given in Exercise 5.18(iii) with $\beta =1-\alpha$ and $\alpha \in (0,\frac12)$ , and let $F_n$ be the empirical distribution based on a random sample from a distribution $F$ .

?

(i)

Let $X_{(1)}\leq \cdots \leq X_{(n)}$ be the order statistics. Show that

T(F_n)=\overline{X}_\alpha =\frac1{(1-2\alpha )n}\sum _{j=m_\alpha +1}^{n-m_\alpha }X_{(j)},

which is called the $\alpha$ -trimmed sample mean. Here $m_\alpha$ is the integer part of $n\alpha$ .

(ii)

Assume that $F$ is continuous at $F^{-1}(\alpha )$ and $F^{-1}(1-\alpha )$ and is symmetric about $\theta$ . Show that

\sigma _\alpha ^2=\frac2{(1-2\alpha )^2}\left\{ \int _0^{F_0^{-1}(1-\alpha )}x^2\, dF_0(x)+\alpha [F_0^{-1}(1-\alpha )]^2\right\} ,

where $F_0(x-\theta )=F(x)$ , is equal to the $\sigma_F^2$ in part (ii) of the previous exercise with $J(t)=(1-2\alpha )^{-1}\; \mathbb {1}_{(\alpha ,1-\alpha )}(t)$ .

(iii)

Show that if $F_0'(0)$ exists and is positive, then $\lim_{\alpha \to \frac12}\sigma_\alpha^2=1/[2F_0'(0)]^2$ .

(iv)

Show that if $\sigma^2=\int x^2\, dF_0(x)<\infty$ , then $\lim_{\alpha \to 0}\sigma_\alpha^2=\sigma^2$ .

Remark.

?

An L-functional is $T(G)=\int xJ(G(x))\, dG(x)$ on the distributions for which the integral is well defined, with $J$ a Borel function on $[0,1]$ . Its influence function is $\phi_F(x)=L_F(\delta_x-F)$ , where $L_F$ is the Gateaux differential and $\delta_x$ is the point mass at $x$ . Here $J(t)=(1-2\alpha )^{-1}\; \mathbb {1}_{(\alpha ,1-\alpha )}(t)$ .

Corresponding exercises in Shao (2003): 5.65, 5.72, 5.73.

Problem 5.22

Calculate $\sigma_F^2$ defined in Exercise 5.20(ii) with $J(t)=4t-2$ and $F$ being the double exponential distribution with location parameter $\mu \in \mathbb {R}$ and scale parameter 1.

?

Remark.

?

The quantity in Exercise 5.20(ii) is $\sigma_F^2=\int_{-\infty }^{\infty }\int_{-\infty }^{\infty }J(F(x))J(F(y))[F(\min \left\{ x,y\right\} )-F(x)F(y)]\, dx\, dy$ .

Corresponding exercise in Shao (2003): 5.75.

Problem 5.23

Let $T(G)=G^{-1}(p)$ be the $p$ th quantile functional. Suppose that $F$ has a positive derivative $F'$ in a neighborhood of $\theta =F^{-1}(p)$ . Show that $T$ is Gateaux differentiable at $F$ and obtain the influence function.

?

Remark.

?

Corresponding exercise in Shao (2003): 5.59.

Problem 5.24

(i)(ii)

Let $F_n$ be the empirical distribution based on a random sample of size $n$ from a distribution $F$ on $\mathbb {R}$ having Lebesgue density $f$ . Let $\varphi_n(t)$ be the Lebesgue density of $p$ th sample quantile $F_n^{-1}(p)$ . Prove that

\varphi _n(t)=n\binom {n-1}{l_p-1}[F(t)]^{l_p-1}[1-F(t)]^{n-l_p}f(t),

where $l_p=np$ if $np$ is an integer and $l_p=1+$ the integer part of $np$ if $np$ is not an integer, by

?

(i)

using the fact that $nF_n(t)$ has a binomial distribution;

(ii)

using the fact that $F_n^{-1}(p)=c_{np}X_{(m_p)}+(1-c_{np})X_{(m_p+1)}$ , where $X_{(j)}$ is the $j$ th order statistic, $m_p$ is the integer part of $np$ , $c_{np}=1$ if $np$ is an integer, and $c_{np}=0$ if $np$ is not an integer.

Remark.

?

Corresponding exercise in Shao (2003): 5.51.

Problem 5.25

Let $F_n$ be the empirical distribution based on a random sample from a distribution $F$ on $\mathbb {R}$ with a finite mean. Show that the $p$ th sample quantile $F_n^{-1}(p)$ has a finite $j$ th moment for sufficiently large $n$ , $j=1,2,\ldots$ , where $p\in (0,1)$ .

?

Remark.

?

Corresponding exercise in Shao (2003): 5.52.

Problem 5.26

Let $F_n$ be the empirical distribution based on a random sample from a distribution $F$ on $\mathbb {R}$ with Lebesgue density $f$ that is positive and continuous at $\theta =F^{-1}(p)$ , $p\in (0,1)$ . Using Scheffe's theorem (e.g., Proposition 1.18 in Shao, 2003), prove that

\sqrt n[F_n^{-1}(p)-\theta ]\to _d\mathscr {N}\left(0, \frac{p(1-p)}{[f(\theta )]^2}\right).

?

Remark.

?

Corresponding exercise in Shao (2003): 5.54.

Problem 5.27

Let $\left\{ k_n\right\}$ be a sequence of integers satisfying $k_n/n=p+o(n^{-1/2})$ with $p\in (0,1)$ , and let $(X_1,\ldots ,X_n)$ be a random sample from a distribution $F$ on $\mathbb {R}$ with $F'(\theta_p)>0$ , where $\theta_p=F^{-1}(p)$ . Let $X_{(j)}$ be the $j$ th order statistic. Show that

\sqrt n(X_{(k_n)}-\theta _p)\to _d\mathscr {N}\left(0, p(1-p)/[F'(\theta _p)]^2\right).

?

Remark.

?

Corresponding exercise in Shao (2003): 5.55.

Problem 5.28

(i)(ii)

Let $G,G_1,G_2,\ldots$ be cumulative distributions on $\mathbb {R}$ . Suppose that $\lim_n\sup_x\left|G_n(x)-G(x)\right|=0$ , $G$ is continuous, and $G^{-1}$ is continuous at $p\in (0,1)$ .

?

(i)

Show that $\lim_nG_n^{-1}(p)=G^{-1}(p)$ .

(ii)

Show that the result in (i) holds for any $p\in (0,1)$ if $G'(x)$ exists and is positive for any $x\in \mathbb {R}$ .

Remark.

?

Corresponding exercise in Shao (2003): 5.112.

Problem 5.29

(i)(ii)(iii)(iv)

Calculate the asymptotic relative efficiency of the Hodges--Lehmann estimator with respect to the sample mean based on a random sample from $F$ when

?

(i)

$F$ is the cumulative distribution of $\mathscr {N}\left(\mu , \sigma^2\right)$ ;

(ii)

$F$ is the cumulative distribution of the logistic distribution with location parameter $\mu$ and scale parameter $\sigma$ ;

(iii)

$F$ is the cumulative distribution of the double exponential distribution with location parameter $\mu$ and scale parameter $\sigma$ ;

(iv)

$F(x)=F_0(x-\mu )$ , where $F_0(x)$ is the cumulative distribution function of the t-distribution $t_\nu$ with $\nu \geq 3$ .

Remark.

?

The Hodges--Lehmann estimator is a median of the values $(X_i+X_j)/2$ , $i,j=1,\ldots ,n$ ; if there are two middle values, any point between them may be used.

Corresponding exercise in Shao (2003): 5.47.

Problem 5.30

(i)(ii)(iii)

Consider a random sample from a distribution $F$ on $\mathbb {R}$ . In each of the following cases, obtain the asymptotic relative efficiency of the sample median with respect to the sample mean.

?

(i)

$F$ is the cumulative distribution function of the uniform distribution on the interval $(\theta -\frac12,\theta +\frac12)$ , $\theta \in \mathbb {R}$ .

(ii)

$F(x)=F_0(x-\theta )$ and $F_0$ is the cumulative distribution function with Lebesgue density $(1+x^2)^{-1}\; \mathbb {1}_{(-c,c)}(x)/\int_{-c}^c(1+x^2)^{-1}\, dt$ .

(iii)

$F(x)=(1-\epsilon )\Phi \left(\frac{x-\mu }\sigma \right)+\epsilon D\left(\frac{x-\mu }\sigma \right)$ , where $\epsilon \in (0,1)$ is a known constant, $\Phi$ is the cumulative distribution function of the standard normal distribution, $D$ is the cumulative distribution function of the double exponential distribution with location parameter 0 and scale parameter 1, and $\mu \in \mathbb {R}$ and $\sigma >0$ are unknown parameters.

Remark.

?

Corresponding exercises in Shao (2003): 5.61, 5.62, 5.63.

Problem 5.31

(i)(ii)(iii)(iv)

Let $(X_1,\ldots ,X_n)$ be a random sample from a distribution on $\mathbb {R}$ with Lebesgue density $2^{-1}(1-\theta^2)e^{\theta x-\left|x\right|}$ , where $\theta \in (-1,1)$ is unknown.

?

(i)

Show that the median of the distribution of $X_1$ is given by $m(\theta )=(1-\theta )^{-1}\log (1+\theta )$ when $\theta \geq 0$ and $m(\theta )=-m(-\theta )$ when $\theta <0$ .

(ii)

Show that the mean of the distribution of $X_1$ is $\mu (\theta )=2\theta /(1-\theta^2)$ .

(iii)

Show that the inverse functions of $m(\theta )$ and $\mu (\theta )$ exist. Obtain the asymptotic relative efficiency of $m^{-1}(\widehat{m})$ with respect to $\mu^{-1}(\overline{X})$ , where $\widehat{m}$ is the sample median and $\overline{X}$ is the sample mean.

(iv)

Is $\mu^{-1}(\overline{X})$ asymptotically efficient in estimating $\theta$ ?

Remark.

?

Corresponding exercise in Shao (2003): 5.64.

Problem 5.32

(i)(ii)

Obtain the asymptotic relative efficiency of the trimmed sample mean $\overline{X}_\alpha$ (Exercise 5.21) with respect to

?

(i)

the sample mean, based on a random sample of size $n$ from the double exponential distribution with location parameter $\theta \in \mathbb {R}$ and scale parameter 1;

(ii)

the sample median, based on a random sample of size $n$ from the Cauchy distribution with location parameter $\theta \in \mathbb {R}$ and scale parameter 1.

Remark.

?

For $0<\alpha <\frac12$ , the trimmed sample mean is $\overline{X}_\alpha =[(1-2\alpha )n]^{-1}\sum_{j=m_\alpha +1}^{n-m_\alpha }X_{(j)}$ , where $m_\alpha$ is the integer part of $n\alpha$ and $X_{(1)}\leq \cdots \leq X_{(n)}$ are the order statistics.

Corresponding exercises in Shao (2003): 5.70, 5.71.

Problem 5.33

(i)(ii)

Let $(X_1,\ldots ,X_n)$ be a random sample from a distribution $F$ on $\mathbb {R}$ that is symmetric about $\theta \in \mathbb {R}$ . Huber's estimator of $\theta$ is defined as a solution of $\sum_{i=1}^n\psi (X_i,t)=0$ , where

\psi (x,t)=\begin{cases} C& t-x>C,\\ t-x& \left|x-t\right|\leq C,\\ -C& t-x<-C\end{cases}

and $C>0$ is a constant. Assume that $F$ is continuous at $\theta -C$ and $\theta +C$ .

?

(i)

Show that the function

\Psi (\gamma )=\int _{\gamma -C}^{\gamma +C}(\gamma -x)\, dF(x)+CF(\gamma -C)-C[1-F(\gamma +C)]

is differentiable at $\theta$ and $\Psi (\theta )=0$ .

(ii)

Show that the asymptotic relative efficiency of Huber's estimator with respect to the sample mean is $\operatorname {Var}\left[X_1\right]/\sigma_F^2$ , where

\sigma _F^2=\frac{\int _{\theta -C}^{\theta +C}(\theta -x)^2\, dF(x)+C^2F(\theta -C)+C^2[1-F(\theta +C)]}{[F(\theta +C)-F(\theta -C)]^2}.

Remark.

?

Corresponding exercise in Shao (2003): 5.85.

Problem 5.34

For Huber's estimator $\widehat{\theta }$ in the previous exercise, obtain a formula $e(F)$ for the asymptotic relative efficiency of $\widehat{\theta }$ with respect to the sample mean, when

F(x)=(1-\epsilon )\Phi \left(\frac{x-\theta }\sigma \right)+\epsilon \Phi \left(\frac{x-\theta }{\tau \sigma }\right),

where $\Phi$ is the cumulative distribution function of $\mathscr {N}\left(0, 1\right)$ , $\sigma >0$ , $\tau >0$ , and $0<\epsilon <1$ . Show that $\lim_{\tau \to \infty }e(F)=\infty$ . Find the value of $e(F)$ when $\epsilon =0$ , $\sigma =1$ , and $C=1.5$ .

?

Remark.

?

Huber's estimator is a root $t=\widehat{\theta }$ of $\sum_{i=1}^n\psi (X_i,t)=0$ , where $\psi (x,t)=\max \left\{ -C,\min \left\{ C,t-x\right\} \right\}$ and $C>0$ . The observations are a random sample from $F$ , symmetric about $\theta$ , with $F$ continuous at $\theta -C$ and $\theta +C$ .

Corresponding exercise in Shao (2003): 5.86.

Problem 5.35

Consider the L-functional $T$ defined in Exercise 5.17. Let $F_n$ be the empirical distribution based on a random sample of size $n$ from a distribution $F$ , $\sigma^2=\int [\phi_F(x)]^2\, dF(x)$ , and $\sigma_{F_n}^2=\int [\phi_{F_n}(x)]^2\, dF_n(x)$ , where $\phi_G$ denotes the influence function of $T$ at distribution $G$ . Show that $\lim_n\sigma_{F_n}^2=\sigma_F^2$ a.s., under one of the following two conditions:

$J$ is bounded, $J(t)=0$ when $t\in [0,\alpha ]\cup [\beta ,1]$ for some constants $\alpha <\beta$ , and the set $D=\left\{ x:J\text{ is discontinuous at }F(x)\right\}$ has Lebesgue measure 0.
$J$ is continuous on $[0,1]$ and $\int x^2\, dF(x)<\infty$ .

?

Remark.

?

An L-functional is $T(G)=\int xJ(G(x))\, dG(x)$ on the distributions for which the integral is well defined, with $J$ a Borel function on $[0,1]$ . Its influence function is $\phi_F(x)=L_F(\delta_x-F)$ , where $L_F$ is the Gateaux differential and $\delta_x$ is the point mass at $x$ .

Corresponding exercise in Shao (2003): 5.99.

Problem 5.36

Let $(X_1,\ldots ,X_n)$ be a random sample from a distribution $F$ on $\mathbb {R}$ and let $U_n$ be a U-statistic (see Exercise 3.25) with kernel $h(x_1,\ldots ,x_m)$ satisfying $\mathbb {E}\left[[h(X_1,\ldots ,X_m)]^2\right]<\infty$ , where $m<n$ . Assume that $\zeta_1=\operatorname {Var}\left[h_1(X_1)\right]>0$ , where $h_1(x)=\mathbb {E}\left[h(x,X_2,\ldots ,X_m)\right]$ . Derive a consistent variance estimator for $U_n$ .

?

Remark.

?

For a symmetric kernel $h$ , the U-statistic in Exercise 3.25 is $U_n=\binom nm^{-1}\sum_{1\leq i_1<\cdots <i_m\leq n}h(X_{i_1},\ldots ,X_{i_m})$ . That exercise gives $\operatorname {Var}\left[U_n\right]=m^2\zeta_1/n+O(n^{-2})$ when $\zeta_1>0$ .

Corresponding exercise in Shao (2003): 5.100.

Problem 5.37

For Huber's estimator defined in Exercise 5.33, derive a consistent estimator of its asymptotic variance $\sigma_F^2$ .

?

Remark.

?

Huber's estimator is a root $t=\widehat{\theta }$ of $\sum_{i=1}^n\psi (X_i,t)=0$ , where $\psi (x,t)=\max \left\{ -C,\min \left\{ C,t-x\right\} \right\}$ and $C>0$ . The observations are a random sample from $F$ , symmetric about $\theta$ , with $F$ continuous at $\theta -C$ and $\theta +C$ .

Corresponding exercise in Shao (2003): 5.101.

Problem 5.38

Let $X_1,\ldots ,X_n$ be random variables. For any estimator $\widehat{\theta }$ , its jackknife variance estimator is defined as

v_J=\frac{n-1}n\sum _{i=1}^n\left(\widehat{\theta }_{-i}-\frac1n\sum _{j=1}^n\widehat{\theta }_{-j}\right)^2,

where $\widehat{\theta }_{-i}$ is the same as $\widehat{\theta }$ but is based on $n-1$ observations $X_1,\ldots ,X_{i-1},X_{i+1},\ldots ,X_n$ , $i=1,\ldots ,n$ . Let $\overline{X}$ be the sample mean and $\widehat{\theta }=\overline{X}^2$ . Show that

v_J=\frac{4\overline{X}^2\widehat{c}_2}{n-1}-\frac{4\overline{X}\widehat{c}_3}{(n-1)^2}+\frac{\widehat{c}_4-\widehat{c}_2^2}{(n-1)^3},

where $\widehat{c}_k=n^{-1}\sum_{i=1}^n(X_i-\overline{X})^k$ , $k=2,3,4$ .

?

Remark.

?

Corresponding exercise in Shao (2003): 5.104.

Problem 5.39

Let $X_1,\ldots ,X_n$ be random variables and $X_1^*,\ldots ,X_n^*$ be a random sample (i.e., a simple random sample with replacement) from $X_1,\ldots ,X_n$ . For any estimator $\widehat{\theta }$ , its bootstrap variance estimator is $v_B=\operatorname {Var}_{*}\left[\widehat{\theta }^*\right]$ , where $\widehat{\theta }^*$ is the same as $\widehat{\theta }$ but is based on $X_1^*,\ldots ,X_n^*$ and $\operatorname {Var}_{*}\left[\, \cdot \, \right]$ is the variance with respect to the distribution of $X_1^*,\ldots ,X_n^*$ , given $X_1,\ldots ,X_n$ . Let $\overline{X}$ be the sample mean and $\widehat{\theta }=\overline{X}^2$ . Show that

v_B=\frac{4\overline{X}^2\widehat{c}_2}n+\frac{4\overline{X}\widehat{c}_3}{n^2}+\frac{\widehat{c}_4-\widehat{c}_2^2}{n^3},

where $\widehat{c}_k=n^{-1}\sum_{i=1}^n(X_i-\overline{X})^k$ , $k=2,3,4$ .

?

Remark.

?

Corresponding exercise in Shao (2003): 5.111.

Problem 5.40

Let $X_1,\ldots ,X_n$ be a random sample from a distribution on $\mathbb {R}^{k}$ with a finite $\operatorname {Var}\left[X_1\right]$ . Let $X_1^*,\ldots ,X_n^*$ be a random sample from $X_1,\ldots ,X_n$ . Show that for almost all given sequences $X_1,X_2,\ldots$ , $\sqrt n(\overline{X}^*-\overline{X})\to_dN_k(0,\operatorname {Var}\left[X_1\right])$ , where $\overline{X}$ is the sample mean based on $X_1,\ldots ,X_n$ and $\overline{X}^*$ is the sample mean based on $X_1^*,\ldots ,X_n^*$ .

?

Remark.

?

Corresponding exercise in Shao (2003): 5.113.