Math Problems Vault

Problem 2.3.1

Find the method-of-moments estimators for the following distributions

?

(a)

$\mathscr {N}\left(a, \sigma^2\right), \; a \in \mathbb {R}, \sigma > 0$ ;

(b)

$\Gamma (\alpha , \beta )$ . Recall that the density of the gamma distribution equals

p_{\alpha , \beta } = \frac{\beta ^\alpha }{\Gamma (\alpha )}x^{\alpha - 1}e^{-\beta x} \; \mathbb {1}_{x \in [0, +\infty )},

where $\alpha , \beta > 0$ . The gamma function: $\Gamma (\alpha ) = \int_0^{\infty } x^{\alpha - 1}e^{-x}dx$ . This is a continuous analogue of the factorial function, namely $\Gamma (n+1) = n!, \; n \in \mathbb {Z}_+$ . In general $\Gamma (\alpha + 1) = \alpha \Gamma (\alpha ), \; \forall \alpha > 0$ . Another useful value: $\Gamma (\frac{1}{2}) = \sqrt{\pi }$ .

(c)

$U[a,b], \; -\infty < a < b < +\infty$

(d)

$\operatorname {Pois}\left(\lambda \right), \; \lambda > 0$ ;

(e)

$\operatorname {Bin}\left(m, p\right), \; m \in \mathbb {N}, p \in (0,1)$ ;

(f)

$\operatorname {Geom}\left(p\right)$ . Recall that $\operatorname {Geom}\left(p\right)$ is a discrete distribution with support $\mathbb {Z}_+$ and the following probability function: $P_{p}(k) = (1-p)^kp, \; k \in \mathbb {Z}_+$ .

(g)

$\operatorname {Beta}(\lambda_1, \lambda_2)$ . Recall that the density of the beta distribution equals

p_{\lambda _1, \lambda _2}(x) = \frac{x^{\lambda _1-1}(1 - x)^{\lambda _2 - 1}}{B(\lambda _1, \lambda _2)} \; \mathbb {1}_{x \in [0,1]},

$\lambda_1, \lambda_2 > 0$ . The beta function $B(\lambda_1, \lambda_2)$ :

B(\lambda _1, \lambda _2) = \int _0^1 x^{\lambda _1 - 1}(1 - x)^{\lambda _2 - 1} dx = \frac{\Gamma (\lambda _1)\Gamma (\lambda _2)}{\Gamma (\lambda _1 + \lambda _2)}

Problem 2.3.2

Find the maximum likelihood estimators for the following distributions:

?

(a)

$\mathscr {N}\left(a, \sigma^2\right)$ in three cases: when only one of the parameters is unknown, and when both parameters are unknown;

(b)

$\Gamma (\alpha , \beta ), \; \alpha ,\beta > 0$ , if the parameter $\alpha$ is known;

(c)

$U[a,b], \; -\infty < a < b < +\infty$

(d)

$\operatorname {Pois}\left(\lambda \right), \; \lambda > 0$ ;

(e)

$\operatorname {Bin}\left(m, p\right), \; m \in \mathbb {N}, p \in (0,1)$ , if the parameter $m$ is known;

(f)

$\operatorname {Geom}\left(p\right), \; p \in (0,1)$ .

(g)

$\operatorname {Pareto}(\theta , 2), \; \theta > 0$ . Recall that the Pareto distribution $\operatorname {Pareto}(x_0, \alpha )$ is a continuous distribution with density

p_{x_0, \alpha }(x) = \frac{\alpha x_0^\alpha }{x^{\alpha + 1}}\; \mathbb {1}_{x \in [x_0, +\infty )}.

$x_0$ is interpreted as the shift of the distribution, and $\alpha$ as the shape parameter.

Problem 2.3.3

Let $X_1, \ldots , X_n$ be a sample from a shifted exponential distribution with density

p_{\alpha , \beta } = \frac{1}{\alpha } e^{\frac{\beta - x}{\alpha }} \; \mathbb {1}_{x \in [\beta , +\infty )},

where $\theta = (\alpha , \beta )^T \in \mathbb {R}_+ \times \mathbb {R}$ . Find the maximum likelihood estimator for $\theta$ . Prove that the resulting estimator $\widehat{\alpha }$ for $\alpha$ is asymptotically normal, and find its asymptotic variance.

?

Problem 2.3.4

(a)(b)

Find the maximum likelihood estimator for the shift parameter $x_0$ in the Cauchy distribution $\operatorname {Cauchy}(x_0, 1)$ , if the sample consists of

?

(a)

one observation,

(b)

two observations.

Recall that the Cauchy distribution $\operatorname {Cauchy}(x_0, \gamma )$ is a continuous distribution with density

p_{\theta , \gamma }(x) = \frac{1}{\pi \gamma \left(1 + \left(\frac{x - x_0}{\gamma }\right)^2\right)} = \frac{\gamma }{\pi \left(\gamma ^2 + (x - x_0)^2\right)}.

$x_0$ is interpreted as the shift of the distribution, and $\gamma$ as the scale.

Problem 2.3.5

Propose an asymptotically normal estimator of the parameter $x_0^2$ in the Cauchy distribution model $\operatorname {Cauchy}(x_0, 1)$ with shift $x_0$ (see problem 3.4), and find its asymptotic variance.

?

Problem 2.3.6

(a)(b)

Find the method-of-moments estimators for the parameters of the following distributions

?

(a)

the distribution $\operatorname {Pareto}(1, \alpha ), \; \alpha > 0$ with density $p_{\alpha }(x) = \frac{\alpha }{x^{\alpha + 1}}\; \mathbb {1}_{x \in [1, +\infty )}$ ;

(b)

the distribution $\operatorname {Cauchy}(0, \gamma ), \; \gamma > 0$ with density $p_{\gamma }(x) = \frac{\gamma }{\pi (\gamma^2 + x^2)}$ .

Problem 2.3.7

Let $X_1 \sim U[0, \theta ], \; \theta > 0$ . Find an unbiased estimator of the parameter $\frac{1}{\theta }$ .

?

Problem 2.3.8

Let $X_1, \ldots , X_n$ be a sample from the distribution $\operatorname {Pois}\left(\lambda \right), \; \lambda > 0$ . Find an unbiased estimator of the parameter $\lambda^3$ .

?

Problem 2.3.9

Let $X_1, \ldots , X_n$ be a sample from the distribution $e^{U[0, \theta ]}, \; \theta > 0$ . I.e. if $\xi \sim U[0, \theta ]$ , then $X_1 \sim e^\xi$ . Find the maximum likelihood estimator of the parameter $\theta$ and check it for consistency.

?

Problem 2.3.10

Let $X_1, \ldots , X_n$ be a sample from the Laplace distribution with density

p_\theta (x) = \frac{1}{2}e^{-\left|x - \theta \right|}

Find the estimator of the parameter $\theta$ by the maximum likelihood method.

?