Point Estimates and Their Properties

Prove Theorem 2.1: let $\hat{\theta }_{n}$ be an estimator of a parameter $\theta \in \mathbb {R}$ such that $\mathbb {E}\left[\left|\Delta \hat{\theta }_{n}\right|^{2}\right] < \infty$, where $\Delta \hat{\theta }_{n} := \hat{\theta }_{n}-\theta$. Then the mean-square error $\Delta_{n} := \mathbb {E}\left[\left|\Delta \hat{\theta }_{n}\right|^{2}\right]$ equals $\Delta_{n} = l_{n}^{2}+d_{n}$, where $l_{n} := \mathbb {E}\left[\Delta \hat{\theta }_{n}\right]$ is the bias of the estimator $\hat{\theta }_{n}$, and $d_{n} := \operatorname {Var}\left[\Delta \hat{\theta }_{n}\right]$ is the variance of its error.

Let $\theta$ be a nonrandom parameter and let $\widehat{\theta }_{n}$ be an unbiased estimator of it. Show that $\Delta_{n} = \operatorname {Var}\left[\widehat{\theta }_{n}\right]$.

The sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ is generated by a random variable $X$ with known mean $m = \mathbb {E}\left[X\right]$ and unknown variance $\theta = \operatorname {Var}\left[X\right]$. Prove that the statistic $S_{n}^{2} = \frac{1}{n} \sum_{k = 1}^{n}\left(X_{k}-m\right)^{2}$ is an unbiased and strongly consistent estimator of the parameter $\theta$.

The sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ corresponds to the distribution $R[0 ; \theta ]$, $\theta > 0$. Show that $X_{(n)}$ is a mean-square consistent estimator of the parameter $\theta$.

The sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ corresponds to the distribution $\operatorname {Exp}\left(\theta \right), \theta > 0$. Prove that $\hat{\theta }_{n} = \sqrt{\frac{2 n}{\sum_{k = 1}^{n} X_{k}^{2}}}$ is a strongly consistent estimator of the parameter $\theta$.

Let the sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ correspond to the normal distribution $\mathcal{N}\left(\theta , \sigma^{2}\right)$, where $\sigma$ is known. Show that the statistic $T_{n} = \left(\bar{X}_{n}\right)^{2}-\frac{\sigma^{2}}{n}$ is an unbiased and strongly consistent estimator of the function $g(\theta ) = \theta^{2}$.

The sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ is generated by the random variable $X \sim R[0 ; \theta ], \theta > 0$. Prove that $\widehat{\theta }_{n} = 2 \bar{X}_{n}$ is an unbiased and strongly consistent estimator of $\theta$.

Let the sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ correspond to the distribution $\operatorname {Bin}\left(N, p\right)$, where $N$ is known. Show that the statistic $T_{n} = \frac{\bar{X}_{n}\left(N-\bar{X}_{n}\right)}{N}$ is an asymptotically unbiased and strongly consistent estimator of the parameter $\theta = \operatorname {Var}\left[X_{1}\right]$.