Math Problems Vault

Problem 4.2.1

(a)(b)

Referring to Example 1.5, suppose that $X$ has the binomial distribution $b(p, n)$ and the family of prior distributions for $p$ is the family of beta distributions $B(a, b)$ .

?

(a)

Show that the marginal distribution of $X$ is the beta-binomial distribution with mass function

\binom {n}{x} \frac{\Gamma (a+b)}{\Gamma (a) \Gamma (b)} \frac{\Gamma (x+a) \Gamma (n-x+b)}{\Gamma (n+a+b)}.

(b)

Show that the mean and variance of the beta-binomial is given by

\mathbb {E}\left[X\right]=\frac{n a}{a+b} \quad \text{ and } \quad \text{ var } X=n\left(\frac{a}{a+b}\right)\left(\frac{b}{a+b}\right)\left(\frac{a+b+n}{a+b+1}\right).

[Hint: For part (b), the identities $\mathbb {E}\left[X\right]=E[\mathbb {E}\left[X \mid p\right]]$ and $\operatorname {var} X=\operatorname {var}[\mathbb {E}\left[X \mid p\right]]+ E[\operatorname {var}(X \mid p)]$ are helpful.]

Problem 4.2.2

(a)(b)(c)

For the situation of Example 2.1, Lindley and Phillips (1976) give a detailed account of the effect of stopping rules, which we can illustrate as follows. Let $X$ be the number of successes in $n$ Bernoulli trials with success probability $p$ .

?

(a)

Suppose that the number of Bernoulli trials performed is a prespecifed number $n$ , so that we have the binomial sampling model, $\mathbb {P}\left(X=x\right)=\binom {n}{x} p^{x}(1-p)^{n-x}$ , $x=0,1, \ldots , n$ . Calculate the Bayes risk of the Bayes estimator (1.1.12) and the UMVU estimator of $p$ .

(b)

Suppose that the number of Bernoulli trials performed is a random variable $N$ . The value $N=n$ was obtained when a prespecified number, $x$ , of successes was observed so that we have the negative binomial sample model, $\mathbb {P}\left(N=n\right)=\binom {n-1}{x-1} p^{x}(1- p)^{n-x}$ , $n=x$ . Calculate the Bayes risk of the Bayes estimator and the UMVU estimator of $p$ .

(c)

Calculate the mean squared errors of all three estimators under each model. If it is unknown which sampling mechanism generated the data, which estimator do you prefer overall?

Problem 4.2.3

(a)(b)(c)

Show that the estimator (2.2.4) tends in probability

?

(a)

to $\theta$ as $n \rightarrow \infty$ ,

(b)

to $\mu$ as $b \rightarrow 0$ , and

(c)

to $\theta$ as $b \rightarrow \infty$ .

Problem 4.2.4

(a)(b)(c)

Bickel and Mallows (1988) further investigate the relationship between unbiasedness and Bayes, specifying conditions under which these properties cannot hold simultaneously. In addition, they show that if a prior distribution is improper, then a posterior mean can be unbiased. Let $X \sim \frac{1}{\theta } f(x / \theta ), x>0$ , where $\int_{0}^{\infty } t f(t) d t=1$ , and let $\pi (\theta )=\frac{1}{\theta^{2}} d \theta$ , $\theta >0$ .

?

(a)

Show that $\mathbb {E}\left[X \mid \theta \right]=\theta$ , so $X$ is unbiased.

(b)

Show that $\pi (\theta \mid x)=\frac{x^{2}}{\theta^{3}} f(x / \theta )$ is a proper density.

(c)

Show that $\mathbb {E}\left[\theta \mid x\right]=x$ , and hence the posterior mean, is unbiased.

Problem 4.2.5

DasGupta (1994) presents an identity relating the Bayes risk to bias, which illustrates that a small bias can help achieve a small Bayes risk. Let $X \sim f(x \mid \theta )$ and $\theta \sim \pi (\theta )$ . The Bayes estimator under squared error loss is $\delta^{\pi }=\mathbb {E}\left[\theta \mid x\right]$ . Show that the Bayes risk of $\delta^{\pi }$ can be written

r\left(\pi , \delta ^{\pi }\right)=\int _{\Theta } \int _{\mathcal{X}}\left[\theta -\delta ^{\pi }(x)\right]^{2} f(x \mid \theta ) \pi (\theta ) d x d \theta =\int _{\Theta } \theta b(\theta ) \pi (\theta ) d \theta

where $b(\theta )=\mathbb {E}\left[\delta^{\pi }(X) \mid \theta \right]-\theta$ is the bias of $\delta^{\pi }$ .

?

Problem 4.2.6

Verify the estimator (2.2.10).

?

Problem 4.2.7

In Example 2.6, verify that the posterior distribution of $\tau$ is $\Gamma (r+g-1 / 2,1 /(\alpha +z))$ .

?

Problem 4.2.8

In Example 2.6 with $\alpha =g=0$ , show that the posterior distribution given the $X$ 's of $\sqrt{n}(\theta -\bar{X}) / \sqrt{Z /(n-1)}$ is Student's $t$ -distribution with $n-1$ degrees of freedom.

?

Problem 4.2.9

In Example 2.6, show that the posterior distribution of $\theta$ is symmetric about $\bar{x}$ when the joint prior of $\theta$ and $\sigma$ is of the form $h(\sigma ) d \sigma d \theta$ , where $h$ is an arbitrary probability density on $(0, \infty )$ .

?

Problem 4.2.10

Rukhin (1978) investigates the situation when the Bayes estimator is the same for every loss function in a certain set of loss functions, calling such estimators universal Bayes estimators. For the case of Example 2.6, using the prior of the form of Problem 2.9, show that $\bar{X}$ is the Bayes estimator under every even loss function.

?

Problem 4.2.11

(a)(b)

Let $X$ and $Y$ be independently distributed according to distributions $P_{\xi }$ and $Q_{\eta }$ , respectively. Suppose that $\xi$ and $\eta$ are real-valued and independent according to some prior distributions $\Lambda$ and $\Lambda^{\prime }$ . If, with squared error loss, $\delta_{\Lambda }$ is the Bayes estimator of $\xi$ on the basis of $X$ , and $\delta_{\Lambda^{\prime }}^{\prime }$ is that of $\eta$ on the basis of $Y$ ,

?

(a)

show that $\delta_{\Lambda^{\prime }}^{\prime }-\delta_{\Lambda }$ is the Bayes estimator of $\eta -\xi$ on the basis of $(X, Y)$ ;

(b)

if $\eta >0$ and $\delta_{\Lambda^{\prime }}^{*}$ is the Bayes estimator of $1 / \eta$ on the basis of $Y$ , show that $\delta_{\Lambda } \cdot \delta_{\Lambda^{\prime }}^{*}$ is the Bayes estimator of $\xi / \eta$ on the basis of $(X, Y)$ .

Problem 4.2.12

For the density (2.2.13) and improper prior $(d \sigma / \sigma ) \cdot \left(d \sigma_{A} / \sigma_{A}\right)$ , show that the posterior distribution of $\left(\sigma , \sigma_{A}\right)$ continues to be improper.

?

Problem 4.2.13

(a)(b)

?

(a)

In Example 2.7, obtain the Jeffreys prior distribution of $(\sigma , \tau )$ .

(b)

Show that for the prior of part (a), the posterior distribution of $(\sigma , \tau )$ is proper.

Problem 4.2.14

Verify the Bayes estimator (2.2.14).

?

Problem 4.2.15

(a)(b)(c)

Let $X \sim N(\theta , 1)$ and $L(\theta , \delta )=(\theta -\delta )^{2}$ .

?

(a)

Show that $X$ is the limit of the Bayes estimators $\delta^{\pi_{n}}$ , where $\pi_{n}$ is $N(0,1)$ . Hence, $X$ is both generalized Bayes and a limit of Bayes estimators.

(b)

For the prior measure $\pi (\theta )=e^{a \theta }, a>0$ , show that the generalized Bayes estimator is $X+a$ .

(c)

For $a>0$ , show that there is no sequence of proper priors for which $\delta^{\pi_{n}} \rightarrow X+a$ .

This example is due to Farrell; see Kiefer 1966. Heath and Sudderth (1989), building on the work of Stone (1976), showed that inferences from this model are incoherent, and established when generalized Bayes estimators will lead to coherent (that is, noncontradictory) inferences. Their work is connected to the theory of "approximable by proper priors," developed by Stein (1965) and Stone (1965, 1970, 1976), which shows when generalized Bayes estimators can be looked upon as Bayes estimators.

Problem 4.2.16

(a)(b)

?

(a)

For the situation of Example 2.8, verify that $\delta (x)=x / n$ is a generalized Bayes estimator.

(b)

If $X \sim N(0,1)$ and $L(\theta , \delta )=(\theta -\delta )^{2}$ , show that $X$ is generalized Bayes under the improper prior $\pi (\theta )=1$ .