Fundamentals of Statistics

Consider the family of double exponential distributions: $\mathcal P=\left\{ 2^{-1}e^{-\left|t-\mu \right|}:-\infty <\mu <\infty \right\}$. Show that $\mathcal P$ is not an exponential family.

A discrete random variable $X$ with

where $\gamma (x)\geq 0$, $\theta >0$, and $c(\theta )=\sum_{x=0}^\infty \gamma (x)\theta^x$, is called a random variable with a power series distribution. Show that

Let $X$ be a random variable having the gamma distribution with shape parameter $\alpha$ and scale parameter $\gamma$, where $\alpha$ is know and $\gamma$ is unknown. let $Y=\sigma \log X$. Show that

Let $(X_1,\ldots ,X_n)$ be a random sample from $\mathscr {N}\left(0, 1\right)$. Show that $X_i^2/\sum_{j=1}^nX_j^2$ and $\sum_{j=1}^nX_j^2$ are independent, $i=1,\ldots ,n$.

Let $X=(X_1,\ldots ,X_n)$ be a random $n$-vector having the multivariate normal distribution $N_n(\mu J,D)$, where $J$ is the $n$-vector of $1$'s,

and $\left|\rho \right|<1$. Show that $\overline{X}=n^{-1}\sum_{i=1}^nX_i$ and $W=\sum_{i=1}^n(X_i-\overline{X})^2$ are independent, $\overline{X}$ has the normal distribution $\mathscr {N}\left(\mu , \frac{1+(n-1)\rho }n\sigma^2\right)$, and $W/[(1-\rho )\sigma^2]$ has the chi-square distribution $\chi_{n-1}^2$.

Let $(X_1,\ldots ,X_n)$ be a random sample from the uniform distribution on the interval $[0,1]$ and let $R=X_{(n)}-X_{(1)}$, where $X_{(i)}$ is the $i$th order statistic. Derive the Lebesgue density of $R$ and show that the limiting distribution of $2n(1-R)$ is the chi-square distribution $\chi_4^2$.

Let $(X_1,\ldots ,X_n)$ be a random sample from the exponential distribution with Lebesgue density $\theta^{-1}e^{(a-x)/\theta }\; \mathbb {1}_{(0,\infty )}(x)$, where $a\in \mathbb {R}$ and $\theta >0$ are parameters. Let $X_{(1)}\leq \cdots \leq X_{(n)}$ be order statistics, $X_{(0)}=0$, and $Z_i=X_{(i)}-X_{(i-1)}$, $i=1,\ldots ,n$. Show that

Let $(X_1,\ldots ,X_n)$ be a random sample from the gamma distribution with shape parameter $\alpha$ and scale parameter $\gamma_x$ and let $(Y_1,\ldots ,Y_n)$ be a random sample from the gamma distribution with shape parameter $\alpha$ and scale parameter $\gamma_y$. Assume that $X_i$'s and $Y_i$'s are independent. Derive the distribution of the statistic $\overline{X}/\overline{Y}$, where $\overline{X}$ and $\overline{Y}$ are the sample means based on $X_i$'s and $Y_i$'s, respectively.

Let $(Y_i,Z_i)$, $i=1,\ldots ,n$, be independent and identically distributed random $2$-vectors. The sample correlation coefficient is defined to be

where $\overline{Y}=n^{-1}\sum_{i=1}^nY_i$, $\overline{Z}=n^{-1}\sum_{i=1}^nZ_i$, $S_Y^2=(n-1)^{-1}\sum_{i=1}^n(Y_i-\overline{Y})^2$, and $S_Z^2=(n-1)^{-1}\sum_{i=1}^n(Z_i-\overline{Z})^2$.

Let $X_1,\ldots ,X_n$ be independent and identically distributed random variables with $\mathbb {E}\left[X_1^4\right]<\infty$, $T=(Y,Z)$, and $T_1=Y/\sqrt Z$, where $Y=n^{-1}\sum_{i=1}^n\left|X_i\right|$ and $Z=n^{-1}\sum_{i=1}^nX_i^2$.

Let $X$ be a sample from $P\in \mathcal P$, where $\mathcal P$ is a family of distributions on the Borel $\sigma$-field on $\mathbb {R}^{n}$. Show that if $T(X)$ is a sufficient statistic for $P\in \mathcal P$ and $T=\psi (S)$, where $\psi$ is measurable and $S(X)$ is another statistic, then $S(X)$ is sufficient for $P\in \mathcal P$.

Let $\mathcal P=\left\{ f_\theta :\theta \in \Theta \right\}$, where $f_\theta$'s are probability densities, $f_\theta (x)>0$ for all $x\in \mathbb {R}$ and, for any $\theta \in \Theta$, $f_\theta (x)$ is continuous in $x$. Let $X_1$ and $X_2$ be independent and identically distributed as $f_\theta$. Show that if $X_1+X_2$ is sufficient for $\theta$, then $\mathcal P$ is an exponential family indexed by $\theta$.

Let $X$ and $Y$ be two random variables such that $Y$ has the binomial distribution with size $N$ and probability $\pi$ and, given $Y=y$, $X$ has the binomial distribution with size $y$ and probability $p$.

Let $X_1,\ldots ,X_n$ be independent and identically distributed random variables having the Lebesgue density

where $\theta =(\mu ,\sigma )\in \Theta =\mathbb {R}\times (0,\infty )$. Show that $\mathcal P=\left\{ P_\theta :\theta \in \Theta \right\}$ is an exponential family, where $P_\theta$ is the joint distribution of $X_1,\ldots ,X_n$, and that the statistic $T=(\sum_{i=1}^nX_i,\sum_{i=1}^nX_i^2,\sum_{i=1}^nX_i^3,\sum_{i=1}^nX_i^4)$ is minimal sufficient for $\theta \in \Theta$.

Let $(X_1,\ldots ,X_n)$ be a random sample of random variables having the Lebesgue density $f_\theta (x)=(2\theta )^{-1}[\; \mathbb {1}_{(0,\theta )}(x)+\; \mathbb {1}_{(2\theta ,3\theta )}(x)]$. Find a minimal sufficient statistic for $\theta \in (0,\infty )$.

Let $(X_1,\ldots ,X_n)$ be a random sample of random variables having the Cauchy distribution with location parameter $\mu$ and scale parameter $\sigma$, where $\mu \in \mathbb {R}$ and $\sigma >0$ are unknown parameters. Show that the vector of order statistics is minimal sufficient for $(\mu ,\sigma )$.

Let $(X_1,\ldots ,X_n)$, $n\ge 2$, be a random sample from a distribution having Lebesgue density $f_{\theta ,j}$, where $\theta >0$, $j=1,2$, $f_{\theta ,1}$ is the density of $\mathscr {N}\left(0, \theta^2\right)$, and $f_{\theta ,2}(x)=(2\theta )^{-1}\mathrm{exp}\left(-\left|x\right|/\theta \right)$. Show that $T=(T_1,T_2)$ is minimal sufficient for $(\theta ,j)$, where $T_1=\sum_{i=1}^nX_i^2$ and $T_2=\sum_{i=1}^n\left|X_i\right|$.

Let $(X_1,\ldots ,X_n)$, $n\ge 2$, be a random sample from a distribution with discrete probability density $f_{\theta ,j}$, where $\theta \in (0,1)$, $j=1,2$, $f_{\theta ,1}$ is the Poisson distribution with mean $\theta$, and $f_{\theta ,2}$ is the binomial distribution with size $1$ and probability $\theta$.

Let $(X_1,\ldots ,X_n)$ be a random sample from a distribution on $\mathbb {R}$ having the Lebesgue density $\theta^{-1}\mathrm{exp}\left(-(x-\theta )/\theta \right)\; \mathbb {1}_{(\theta ,\infty )}(x)$, where $\theta >0$ is an unknown parameter.

Let $T$ be a complete (or boundedly complete) and sufficient statistic. Suppose that there is a minimal sufficient statistic $S$. Show that $T$ is minimal sufficient and $S$ is complete (or boundedly complete).

Let $X$ be a discrete random variable with probability density

where $\theta \in (0,1)$. Show that $X$ is boundedly complete, but not complete.

Let $X$ be a discrete random variable with

where $n$ and $N$ are positive integers, $N\ge n$, and $\theta =0,1,\ldots ,N$. Show that $X$ is complete.

Let $X$ be a random variable having the uniform distribution on the interval $(\theta ,\theta +1)$, $\theta \in \mathbb {R}$. Show that $X$ is not complete.

Let $(X_1,\ldots ,X_n)$ be a random sample from the $\mathscr {N}\left(\theta , \theta^2\right)$ distribution, where $\theta >0$ is a parameter. Find a minimal sufficient statistic for $\theta$ and show whether it is complete.

Suppose that $(X_1,Y_1),\ldots ,(X_n,Y_n)$ are independent and identically distributed random $2$-vectors and $X_i$ and $Y_i$ are independently distributed as $\mathscr {N}\left(\mu , \sigma_X^2\right)$ and $\mathscr {N}\left(\mu , \sigma_Y^2\right)$, respectively, with $\theta =(\mu ,\sigma_X^2,\sigma_Y^2)\in \mathbb {R}\times (0,\infty )\times (0,\infty )$. Let $\overline{X}$ and $S_X^2$ be the sample mean and variance for $X_i$'s and $\overline{Y}$ and $S_Y^2$ be the sample mean and variance for $Y_i$'s. Show that $T=(\overline{X},\overline{Y},S_X^2,S_Y^2)$ is minimal sufficient for $\theta$ but $T$ is not boundedly complete.

Suppose that $(X_1,Y_1),\ldots ,(X_n,Y_n)$ are independent and identically distributed random $2$-vectors having the normal distribution with $\mathbb {E}\left[X_1\right]=\mathbb {E}\left[Y_1\right]=0$, $\operatorname {Var}\left[X_1\right]=\operatorname {Var}\left[Y_1\right]=1$, and $\operatorname {Cov}\left[ X_1, Y_1 \right]=\theta \in (-1,1)$.

Let $(X_1,\ldots ,X_n)$, $n>2$, be a random sample from the exponential distribution on $(a,\infty )$ with scale parameter $\theta$. Show that

Let $(X_1,\ldots ,X_n)$, $n>2$, be a random sample of random variables having the uniform distribution on the interval $[a,b]$, where $-\infty <a<b<\infty$. Show that $Z_i=(X_{(i)}-X_{(1)})/(X_{(n)}-X_{(1)})$, $i=2,\ldots ,n-1$, are independent of $(X_{(1)},X_{(n)})$ for any $a$ and $b$, where $X_{(j)}$ is the $j$th order statistic.

Let $(X_1,\ldots ,X_n)$, $n>2$, be a random sample from a distribution $P$ on $\mathbb {R}$ with $\mathbb {E}\left[X_1^2\right]<\infty$, $\overline{X}$ be the sample mean, $X_{(j)}$ be the $j$th order statistic, and $T=(X_{(1)}+X_{(n)})/2$. Consider the estimation of a parameter $\theta \in \mathbb {R}$ under the squared error loss.

Let $(X_1,\ldots ,X_n)$ be a random sample of binary random variables with $\mathbb {P}\left(X_i=1\right)=\theta \in (0,1)$. Consider estimating $\theta$ with the squared error loss. Calculate the risks of the following estimators:

Consider the estimation of an unknown parameter $\theta \ge 0$ under the squared error loss. Show that if $T$ and $U$ are two estimators such that $T\le U$ and $R_T(P)<R_U(P)$, then $R_{T_+}(P)<R_{U_+}(P)$, where $R_T(P)$ is the risk of an estimator $T$ and $T_+$ denotes the positive part of $T$.

Consider the estimation of an unknown parameter $\theta \in \mathbb {R}$ under the squared error loss. Show that if $T$ and $U$ are two estimators such that $\mathbb {P}\left(\theta -t<T<\theta +t\right)\ge \mathbb {P}\left(\theta -t<U<\theta +t\right)$ for any $t>0$, then $R_T(P)\le R_U(P)$.

Let $(X_1,\ldots ,X_n)$ be a random sample from the exponential distribution on $(0,\infty )$ with scale parameter $\theta \in (0,\infty )$. Consider the hypotheses $H_0:\theta \le \theta_0$ versus $H_1:\theta >\theta_0$, where $\theta_0>0$ is a fixed constant. Obtain the risk function (in terms of $\theta$) of the test rule $T_c=\; \mathbb {1}_{(c,\infty )}(\overline{X})$ under the $0$-$1$ loss, where $\overline{X}$ is the sample mean and $c>0$ is a constant.

Consider an estimation problem with a parametric family $\mathcal P=\left\{ P_\theta :\theta \in \Theta \right\}$ and the squared error loss. If $\theta_0\in \Theta$ satisfies that $P_\theta \ll P_{\theta_0}$ for any $\theta \in \Theta$, show that the estimator $T\equiv \theta_0$ is admissible.

Let $(X_1,\ldots ,X_n)$ be a random sample of random variables with $\mathbb {E}\left[X_1^2\right]<\infty$. Consider estimating $\mu =\mathbb {E}\left[X_1\right]$ under the squared error loss. Show that

Consider an estimation problem with $\vartheta \in [c,d]\subset \mathbb {R}$, where $c$ and $d$ are known. Suppose that the action space contains $[c,d]$ and the loss function is $L(\left|\vartheta -a\right|)$, where $L(\cdot )$ is an increasing function on $[0,\infty )$. Show that any decision rule $T$ with $\mathbb {P}\left(T(X)\notin [c,d]\right)>0$ for some $P\in \mathcal P$ is inadmissible.

Let $X$ be a sample from $P\in \mathcal P$, $\delta_0(X)$ be a nonrandomized rule in a decision problem with $\mathbb {R}^{k}$ as the action space, and $T$ be a sufficient statistic for $P\in \mathcal P$. Show that if $\mathbb {E}\left[ \; \mathbb {1}_{A}(\delta_0(X)) \mid T \right]$ is a nonrandomized rule, i.e., $\mathbb {E}\left[ \; \mathbb {1}_{A}(\delta_0(X)) \mid T \right]=\; \mathbb {1}_{A}(h(T))$ for any Borel $A\subset \mathbb {R}^{k}$, where $h$ is a Borel function, then $\delta_0(X)=h(T(X))$ a.s.$P$.

Let $X$ be a sample from $P\in \mathcal P$, $\delta_0(X)$ be a decision rule (which may be randomized) in a problem with $\mathbb {R}^{k}$ as the action space, and $T$ be a sufficient statistic for $P\in \mathcal P$. For any Borel $A\subset \mathbb {R}^{k}$, define

Let $L(P,a)$ be a loss function. Show that

Let $X_1,\ldots ,X_n$ be random variables with a finite common mean $\mu =\mathbb {E}\left[X_i\right]$ and finite variances. Consider the estimation of $\mu$ under the squared error loss.

Let $X$ be a discrete random variable with