Statistical Estimation of Parameters

Find the distribution function of the $k$-th order statistic $X_{(k)}$, $k = 1, \ldots , n$.

The sample corresponds to the distribution $R[0 ; 1]$. Compute $\mathbb {E}\left[X_{(n)}\right]$ and $\operatorname {Var}\left[X_{(n)}\right]$.

The sample corresponds to a distribution $F(x)$ with finite moment $\nu_{r}$. Prove that $\bar{\mu }_{r}(n) \xrightarrow {\text{ a.s. }} \mu_{r}, n \rightarrow \infty$.

A sample of size $n \gg 1$ corresponds to the distribution $\mathcal{N}\left(0 ; \sigma^{2}\right)$. Find the distribution of the sample moment $\bar{\nu }_{2}(n)$ as $n \rightarrow \infty$.

A two-dimensional sample of size $n \gg 1$ corresponds to the distribution $\mathcal{N}(\mu ; K)$, where the covariance matrix is $K = \left[\begin{array}{cc}D_{X} & k_{X Y} \\ k_{Y X} & D_{Y}\end{array}\right]$. Prove that $\sqrt{n}\left(\widehat{k}_{X Y}(n)-k_{X Y}\right) \xrightarrow {d} \xi \sim \mathcal{N}\left(0 ; D_{X} D_{Y}+k_{X Y}^{2}\right)$ as $n \rightarrow \infty$.

The sample corresponds to the distribution $\operatorname {Exp}\left(\lambda \right), \lambda > 0$. Find the limit to which $\bar{\nu }_{2}(n)$ converges almost surely as $n \rightarrow \infty$.

A sample of size $n \gg 1$ is generated by a random variable $X \sim \operatorname {Exp}\left(1\right)$. Estimate $\mathbb {P}\left(\left|\widehat{F}_{n}(1)-F_{X}(1)\right| \leqslant \frac{1}{\sqrt{n}}\right)$.

Prove Theorem 2.1: let $\hat{\theta }_{n}$ be an estimator of a parameter $\theta \in \mathbb {R}$ such that $\mathbb {E}\left[\left|\Delta \hat{\theta }_{n}\right|^{2}\right] < \infty$, where $\Delta \hat{\theta }_{n} := \hat{\theta }_{n}-\theta$. Then the mean-square error $\Delta_{n} := \mathbb {E}\left[\left|\Delta \hat{\theta }_{n}\right|^{2}\right]$ equals $\Delta_{n} = l_{n}^{2}+d_{n}$, where $l_{n} := \mathbb {E}\left[\Delta \hat{\theta }_{n}\right]$ is the bias of the estimator $\hat{\theta }_{n}$, and $d_{n} := \operatorname {Var}\left[\Delta \hat{\theta }_{n}\right]$ is the variance of its error.

Let $\theta$ be a nonrandom parameter and let $\widehat{\theta }_{n}$ be an unbiased estimator of it. Show that $\Delta_{n} = \operatorname {Var}\left[\widehat{\theta }_{n}\right]$.

The sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ is generated by a random variable $X$ with known mean $m = \mathbb {E}\left[X\right]$ and unknown variance $\theta = \operatorname {Var}\left[X\right]$. Prove that the statistic $S_{n}^{2} = \frac{1}{n} \sum_{k = 1}^{n}\left(X_{k}-m\right)^{2}$ is an unbiased and strongly consistent estimator of the parameter $\theta$.

The sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ corresponds to the distribution $R[0 ; \theta ]$, $\theta > 0$. Show that $X_{(n)}$ is a mean-square consistent estimator of the parameter $\theta$.

The sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ corresponds to the distribution $\operatorname {Exp}\left(\theta \right), \theta > 0$. Prove that $\hat{\theta }_{n} = \sqrt{\frac{2 n}{\sum_{k = 1}^{n} X_{k}^{2}}}$ is a strongly consistent estimator of the parameter $\theta$.

Let the sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ correspond to the normal distribution $\mathcal{N}\left(\theta , \sigma^{2}\right)$, where $\sigma$ is known. Show that the statistic $T_{n} = \left(\bar{X}_{n}\right)^{2}-\frac{\sigma^{2}}{n}$ is an unbiased and strongly consistent estimator of the function $g(\theta ) = \theta^{2}$.

The sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ is generated by the random variable $X \sim R[0 ; \theta ], \theta > 0$. Prove that $\widehat{\theta }_{n} = 2 \bar{X}_{n}$ is an unbiased and strongly consistent estimator of $\theta$.

Let the sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ correspond to the distribution $\operatorname {Bin}\left(N, p\right)$, where $N$ is known. Show that the statistic $T_{n} = \frac{\bar{X}_{n}\left(N-\bar{X}_{n}\right)}{N}$ is an asymptotically unbiased and strongly consistent estimator of the parameter $\theta = \operatorname {Var}\left[X_{1}\right]$.

Prove that the parameter estimators constructed in Examples 3.1 and 3.3 are asymptotically unbiased and strongly consistent.

A sample of size $n$ is generated by the random variable $X \sim \operatorname {Exp}\left(\theta \right), \theta > 0$. Find the ML estimator of the parameter $\theta$ and prove its strong consistency.

A sample of size $n$ corresponds to the Poisson distribution $\operatorname {Pois}\left(\theta \right)$, $\theta > 0$. Find the ML estimator of $\theta$ and prove its unbiasedness, strong consistency, and asymptotic normality.

The sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ is generated by the random variable $X \sim E(\theta_{1}, \theta_{2}), \theta_{2} > 0$, i.e.

Find estimators of the parameters $\theta_{1}$ and $\theta_{2}$ by the method of moments. Prove the strong consistency of the obtained estimators.

The sample $Z_{n}$ corresponds to the Rayleigh distribution with distribution function $F(x ; \theta ) = 1-\exp \left\{ -\frac{x^{2}}{\theta }\right\} , x \geqslant 0, \theta > 0$. Find the ML estimator of the parameter $\theta$.

The sample $Z_{n} = \left\{ X_{k}, k = 1, \ldots , n\right\}$ of size $n = 2 m+1$ ($m$ is a natural number) corresponds to the Laplace distribution with density $p(x ; \theta ) = \frac{1}{2} \exp \left\{ -\left|x-\theta \right|\right\}$. Find the ML estimator of the parameter $\theta$.

Under the conditions of Problem 6, for the case $n = 2 m$, show that any statistic of the form $\widehat{\theta }_{n} = (1-\lambda ) X_{(m)}+\lambda X_{(m+1)}, \lambda \in [0 ; 1]$ is an ML estimator of $\theta$.

Let $\widehat{\theta }_{n}$ be the ML estimator of the parameter $\theta$ of the Bernoulli distribution $\operatorname {Bin}\left(1, \theta \right)$. Show that the sequence $\sqrt{n}\left(\widehat{\theta }_{n}-\theta \right)$ is asymptotically normal with parameters ($0 ; \theta (1-\theta )$).

The sample $Z_{n}$ corresponds to the normal distribution with parameters $(\sqrt{\theta } ; 2), \theta \geqslant 0$. Find the ML estimator of $\theta$.

The sample $Z_{n} = \left\{ X_{k}, k = 1, \ldots , n\right\}$ corresponds to the lognormal distribution with parameters $\left(\theta_{1} ; \theta_{2}\right)$, i.e. $\ln X_{k} \sim \mathcal{N}\left(\theta_{1} ; \theta_{2}\right)$. Find the ML estimator of the parameter $\theta = \mathbb {E}\left[X_{k}\right]$ and prove its strong consistency.

Find the ML estimator of the parameter $\theta = \mathbb {E}\left[X_{k}\right]$ from the sample $\left\{ X_{k}, k = 1, \ldots , n\right\}$ corresponding to the distribution $R\left[\theta_{1}, \theta_{2}\right]$.

The sample corresponds to the distribution $\operatorname {Bin}\left(N, \theta \right), \theta \in (0 ; 1)$. Verify the regularity conditions, find $i(\theta )$, and prove the efficiency of the ML estimator of the parameter $\theta$.

Show that the Poisson distribution $\operatorname {Pois}\left(\theta \right)$ is regular. Find $i(\theta )$. Prove the efficiency of the ML estimator $\widehat{\theta }_{n}$ of the parameter $\theta$.

For the distribution $\mathcal{N}\left(\mu ; \theta^{2}\right), \theta > 0$, where $\mu$ is known, find the Fisher information $i(\theta )$.

Verify the regularity of the distribution $\operatorname {Exp}\left(\theta \right), \theta > 0$, and compute $I_{n}(\theta )$. Prove that the estimator $\widetilde{\theta }_{n} = \frac{n-1}{n} \widehat{\theta }_{n}$ is asymptotically efficient if $\widehat{\theta }_{n}$ is the ML estimator of $\theta$.

Show that the information $I_{n}(\theta )$ contained in a sample $Z_{n}$ corresponding to the Laplace distribution (4.10) equals $\frac{n}{\lambda^{2}}$.

Compare in accuracy the estimator $\theta_{n}^{*}$ of the parameter $\theta$ of the distribution $R[0 ; \theta ]$, $\theta > 0$, obtained by the method of moments, with the estimator $\tilde{\theta }_{n}$ considered in Example 4.5.

Let the sample correspond to the distribution $\mathcal{N}(\mu ; \theta ), \theta > 0, \mu$ is known. Prove that the ML estimator of the variance $\theta$ is efficient.

The sample $Z_{n}$ corresponds to the distribution $\mathcal{N}\left(\theta_{1} ; \theta_{2}^{2}\right)$. Find the Fisher information matrix $I\left(\theta_{1}, \theta_{2}\right)$.

For the estimators constructed in Problems 1, 2 and 7, find their representation (4.6) in terms of the sample score.

The sample $\left\{ X_{1}, \ldots , X_{n}\right\} , n \gg 1$ corresponds to the Poisson distribution $\operatorname {Pois}\left(\theta \right), \theta > 0$. Construct an asymptotic confidence interval for $\theta$ of confidence level $q$.

The sample $\left\{ X_{1}, \ldots , X_{n}\right\}$ corresponds to the distribution $\mathcal{N}(\mu ; \theta ), \mu$ is known. Construct a central confidence interval of confidence level $q$ for the variance $\theta$.

The sample $\left\{ X_{1}, \ldots , X_{n}\right\} , n \gg 1$ corresponds to the distribution $\operatorname {Exp}\left(\frac{1}{\theta }\right)$, $\theta > 0$. Construct an asymptotic confidence interval of confidence level $q$ for the parameter $\theta$.

From the sample $z_{n} = \left\{ -0,26 ;-0,36 ; 1,83 ; 0,54 ;-2,06\right\}$ corresponding to the distribution $\mathcal{N}\left(\theta_{1} ; \theta_{2}^{2}\right)$, find confidence intervals of confidence level $q = 0,9$ for $\theta_{1}$ and $\theta_{2}$.

The sample $Z_{n} = \left\{ X_{k}, k = 1, \ldots , n\right\} , n \gg 1$ corresponds to the uniform distribution $R[0 ; a], a > 0$. Construct an asymptotic confidence interval of confidence level $q$ for the parameter $\theta = \mathbb {E}\left[X_{1}\right]$.

The sample $Z_{n} = \left\{ X_{1}, \ldots , X_{n}\right\} , n \gg 1$ corresponds to the distribution with probability density $p(x ; \theta ) = \exp \left\{ \theta -x\right\} , x \geqslant \theta$, where $\theta > 0$. Show that $\left[X_{(1)}+\frac{\ln (1-q)}{n} ; X_{(1)}\right]$ is a confidence interval of confidence level $q$ for $\theta$.

Under the conditions of Problem 6, construct a central confidence interval of confidence level $q$ for $\theta$.

Let the sample $\left\{ X_{1}, \ldots , X_{n}\right\} , n \gg 1$ correspond to the distribution $R[0 ; \theta ], \theta > 0$. Show that $\left(\frac{X_{(n)}}{\theta }\right)^{n}$ is a pivotal statistic, and construct a confidence interval for $\theta$ of minimum length with confidence level $q$.

Let $Z_{n}^{(1)} = \left\{ X_{1}, \ldots , X_{n}\right\}$ and $Z_{n}^{(2)} = \left\{ Y_{1}, \ldots , Y_{n}\right\}$ be two independent samples, where $Z_{n}^{(1)}$ corresponds to the distribution $\mathcal{N}\left(\theta_{1} ; \sigma_{1}^{2}\right)$, and $Z_{n}^{(2)}$ to $\mathcal{N}\left(\theta_{2} ; \sigma_{2}^{2}\right)$ ($\sigma_{1}$ and $\sigma_{2}$ are known). It is required to construct a confidence interval of confidence level $q$ for the parameter $\theta = \theta_{1}-\theta_{2}$.

Generalize the result of Example 6.1 to the case where the variance $\sigma^{2}$ is unknown.

The sample $Z_{n}$ corresponds to the distribution $\mathcal{N}\left(\theta_{1} ; \theta_{2}\right)$. Test, at significance level $p$, the hypothesis $H_{0}: \theta_{2} = \sigma^{2}$ against $H_{1}: \theta_{2} \neq \sigma^{2}$.

Let $\left\{ X_{k}, k = 1, \ldots , n\right\}$ and $\left\{ Y_{m}, m = 1, \ldots , n\right\}$ be independent samples generated by the random variables $X \sim \mathcal{N}\left(\theta_{1} ; \sigma_{1}^{2}\right)$ and $Y \sim \mathcal{N}\left(\theta_{2} ; \sigma_{2}^{2}\right)$, where $\sigma_{1}$ and $\sigma_{2}$ are known. At significance level $p$, test the hypothesis $H_{0}: \theta_{1} = \theta_{2}$ against $H_{1}: \theta_{1} \neq \theta_{2}$.

Under the conditions of Example 6.4, find the probability $\beta$ of a type II error.

Under the conditions of the previous problem, determine the minimum sample size $n_{0}$ for which the quantity $\beta$ will be at most 0,01 , if $\theta_{0} = 0, \gamma = 1$, $\sigma^{2} = 1, p = 0,05$.

Under the conditions of Example 6.4, find the minimum sample size $n_{0}$ for which the probabilities of type I and type II errors do not exceed the given values $a > 0$ and $b > 0$, respectively.

The sample $Z_{n} = \left\{ X_{k}, k = 1, \ldots , n\right\}$ corresponds to the distribution $\mathcal{N}\left(0 ; \theta^{2}\right), \theta > 0$. Construct the most powerful test for testing, at significance level $p$, the hypothesis $H_{0}: \theta = \theta_{0}$ against the alternative $H_{1}: \theta = \sigma > \theta_{0}$.