Generalized linear models

[4/0%]

LaTeX

Show LaTeX

Problem 6.4.16 from Lehmann E.L., Casella G. "Theory of Point Estimation" [LeCa98]More problems in Section 6.4 "Likelihood Estimation: Multiple Roots"

Maximum likelihood estimation in the probit model of Section 3.6 can be implemented using the EM algorithm. We observe independent Bernoulli variables $X_{1}, \ldots , X_{n}$ , which depend on unobservable variables $Z_{i}$ distributed independently as $N\left(\zeta_{i}, \sigma^{2}\right)$ , where

X_{i}= \begin{cases} 0 & \text{ if } Z_{i} \leq u \\ 1 & \text{ if } Z_{i}>u.\end{cases}

Assuming that $u$ is known, we are interested in obtaining ML estimates of $\zeta$ and $\sigma^{2}$ .

(a)

Show that the likelihood function is $p^{\Sigma x_{i}}(1-p)^{n-\Sigma x_{i}}$ , where

p=\mathbb {P}\left(Z_{i}>u\right)=\Phi \left(\frac{\zeta -u}{\sigma }\right).

(b)

If we consider $Z_{1}, \ldots , Z_{n}$ to be the complete data, the complete-data likelihood is

\prod _{i=1}^{n} \frac{1}{\sqrt{2 \pi } \sigma } e^{-\frac{1}{2 \sigma ^{2}}\left(z_{i}-\zeta \right)^{2}}

and the expected complete-data log likelihood is

-\frac{n}{2} \log \left(2 \pi \sigma ^{2}\right)-\frac{1}{2 \sigma ^{2}} \sum _{i=1}^{n}\left[\mathbb {E}\left[Z_{i}^{2} \mid X_{i}\right]-2 \zeta \mathbb {E}\left[Z_{i} \mid X_{i}\right]+\zeta ^{2}\right].

(c)

Show that the EM sequence is given by

\begin{align} & \hat{\zeta }_{(j+1)}=\frac{1}{n} \sum _{i=1}^{n} t_{i}\left(\hat{\zeta }_{(j)}, \hat{\sigma }_{(j)}^{2}\right) \\ & \hat{\sigma }_{(j+1)}^{2}=\frac{1}{n}\left[\sum _{i=1}^{n} v_{i}\left(\hat{\zeta }_{(j)}, \hat{\sigma }_{(j)}^{2}\right)-\frac{1}{n}\left(\sum _{i=1}^{n} t_{i}\left(\hat{\zeta }_{(j)}, \hat{\sigma }_{(j)}^{2}\right)\right)^{2}\right] \end{align}

where

t_{i}\left(\zeta , \sigma ^{2}\right)=\mathbb {E}\left[Z_{i} \mid X_{i}, \zeta , \sigma ^{2}\right] \quad \text{ and } \quad v_{i}\left(\zeta , \sigma ^{2}\right)=\mathbb {E}\left[Z_{i}^{2} \mid X_{i}, \zeta , \sigma ^{2}\right].

(d)

Show that

\begin{align} & \mathbb {E}\left[Z_{i} \mid X_{i}, \zeta , \sigma ^{2}\right]=\zeta +\sigma H_{i}\left(\frac{u-\zeta }{\sigma }\right) \\ & \mathbb {E}\left[Z_{i}^{2} \mid X_{i}, \zeta , \sigma ^{2}\right]=\zeta ^{2}+\sigma ^{2}+\sigma (u+\zeta ) H_{i}\left(\frac{u-\zeta }{\sigma }\right) \end{align}

where

H_{i}(t)= \begin{cases} \frac{\varphi (t)}{1-\Phi (t)} & \text{ if } X_{i}=1 \\ -\frac{\varphi (t)}{\Phi (t)} & \text{ if } X_{i}=0\end{cases}

(e)

Show that $\hat{\zeta }_{(j)} \rightarrow \hat{\zeta }$ and $\hat{\sigma }_{(j)}^{2} \rightarrow \hat{\sigma }^{2}$ , the ML estimates of $\zeta$ and $\sigma^{2}$ .

Problem 6.7.6 from Lehmann E.L., Casella G. "Theory of Point Estimation" [LeCa98]More problems in Section 6.7 "Extensions"

Show that the likelihood equations (7.11) have at most one solution.

Problem 6.7.7 from Lehmann E.L., Casella G. "Theory of Point Estimation" [LeCa98]More problems in Section 6.7 "Extensions"

In Example 7.6, suppose that $p_{i}=1-F\left(\alpha +\beta t_{i}\right)$ and that both $\log F(x)$ and $\log [1- F(x)]$ are strictly concave. Then, the likelihood equations have at most one solution.

Problem 6.7.8 from Lehmann E.L., Casella G. "Theory of Point Estimation" [LeCa98]More problems in Section 6.7 "Extensions"

(a)(b)

(a)

If the cdf $F$ is symmetric and if $\log F(x)$ is strictly concave, so is $\log [1-F(x)]$ .

(b)

Show that $\log F(x)$ is strictly concave when $F$ is strongly unimodal but not when $F$ is Cauchy.

References

[LeCa98] Lehmann E.L., Casella G. Theory of Point Estimation. 2nd ed., Springer, 1998.