Math Problems Vault

Problem 10f.3 from Seber G.A., Lee A.J. "Linear Regression Analysis" [SeLe03]More problems in Section 10f "10.7 Diagnosing Collinearity: 10.7.1 Drawbacks of Centering, 10.7.2 Detection of Points Influencing Collinearity, 10.7.3 Remedies for Collinearity"

Prove that with the priors of Section 10.7.3 , the posterior mean of $\gamma$ is just the ridge estimate (10.64).

?

Problem 10f.4 from Seber G.A., Lee A.J. "Linear Regression Analysis" [SeLe03]More problems in Section 10f "10.7 Diagnosing Collinearity: 10.7.1 Drawbacks of Centering, 10.7.2 Detection of Points Influencing Collinearity, 10.7.3 Remedies for Collinearity"

Prove that with the priors of Section 10.7.3 , the prior distribution of $\widehat{\mathbf{\alpha }}$ is $N_{p-1}\left[\mathbf{0}, \sigma^{2}\left(\mathbf{\Lambda }^{-1}+k^{-1} \mathbf{I}_{p-1}\right)\right]$ .

?

Problem 12b.4 from Seber G.A., Lee A.J. "Linear Regression Analysis" [SeLe03]More problems in Section 12b "12.3 Choosing the Best Subset: 12.3.1 Goodness-Of-Fit Criteria, 12.3.2 Criteria Based on Prediction Error, 12.3.3 Estimating Distributional Discrepancies, 12.3.4 Approximating Posterior Probabilities"

Obtain (12.35) using conditional probability arguments.

?

Problem 12d.1 from Seber G.A., Lee A.J. "Linear Regression Analysis" [SeLe03]More problems in Section 12d "12.5 Shrinkage Methods: 12.5.1 Stein Shrinkage, 12.5.2 Ridge Regression, 12.5.3 Garrote and Lasso Estimates"

Verify that the marginal distribution of $\mathbf{Y}$ in the Bayesian setup of Section 12.5.1 is $N_{p}\left(\mathbf{0},\left(\sigma^{2}+\sigma_{0}^{2}\right) \mathbf{I}_{p}\right)$ .

?

Problem 12e.1 from Seber G.A., Lee A.J. "Linear Regression Analysis" [SeLe03]More problems in Section 12e "12.6 Bayesian Methods: 12.6.1 Predictive Densities, 12.6.2 Bayesian Prediction, 12.6.3 Bayesian Model Averaging"

Verify (12.65). Hint: Use the partitioned matrix inverse formula A.9.1 and multiply out the quadratic form.

?

Problem 12e.2 from Seber G.A., Lee A.J. "Linear Regression Analysis" [SeLe03]More problems in Section 12e "12.6 Bayesian Methods: 12.6.1 Predictive Densities, 12.6.2 Bayesian Prediction, 12.6.3 Bayesian Model Averaging"

Find the form of the posterior predictive density for straight-line regression. Assume that the explanatory variable has been centered and that the prior mean is zero. Also assume that you are predicting a single value (i.e., assume that $m = 1$ ).

?

Problem 3m.1 from Seber G.A., Lee A.J. "Linear Regression Analysis" [SeLe03]More problems in Section 3m "3.12 Bayesian Estimation"

Derive equations (3.60) and (3.61).

?

Problem 3m.2 from Seber G.A., Lee A.J. "Linear Regression Analysis" [SeLe03]More problems in Section 3m "3.12 Bayesian Estimation"

Using the noninformative prior for $\theta$ , show that the conditional posterior density $f(\beta \mid \mathbf{y}, \sigma )$ is multivariate normal. Hence deduce that the posterior mean of $\mathbf{\beta }$ is $\widehat{\mathbf{\beta }}$ .

?

Problem 3m.3 from Seber G.A., Lee A.J. "Linear Regression Analysis" [SeLe03]More problems in Section 3m "3.12 Bayesian Estimation"

(a)(b)(c)(d)

Suppose that we use the noninformative prior for $\mathbf{\theta }$ .

?

(a)

If $v = \sigma^{2}$ , show that $f(v) \propto 1 / v$ .

(b)

Obtain an expression proportional to $f(\mathbf{\beta }, v \mid \mathbf{y})$ .

(c)

Using (3.62), integrate out $\beta$ to obtain

f(v \mid \mathbf{y}) \propto v^{-(\nu / 2+1)} \exp \left(-\frac{a}{v}\right)

where $\nu = n-p$ and $a = \left\| \mathbf{y}-\mathbf{X} \widehat{\mathbf{\beta }}\right\|^{2} / 2$ .

(d)

Find the posterior mean of $v$ .

Problem 4.20 from Shao J. "Mathematical Statistics: Exercises and Solutions" [Sha05]More problems in Chapter 4 "Estimation in Parametric Models"

(i)(ii)(iii)(iv)

Consider the linear model with observed vector $X$ having distribution $N_n(Z\beta ,\sigma^2I_n)$ , where $Z$ is an $n\times p$ known matrix, $p<n$ , $\beta \in \mathbb {R}^{p}$ , and $\sigma^2>0$ .

?

(i)

Assume that $\sigma^2$ is known. Derive the posterior distribution of $\beta$ when the prior distribution for $\beta$ is $N_p(\beta_0,\sigma^2V)$ , where $\beta_0\in \mathbb {R}^{p}$ is known and $V$ is a known positive definite matrix, and find the Bayes estimator of $l^\tau \beta$ under the squared error loss, where $l\in \mathbb {R}^{p}$ is known.

(ii)

Show that the Bayes estimator in (i) is admissible and consistent as $n\to \infty$ , assuming that the minimum eigenvalue of $Z^\tau Z\to \infty$ .

(iii)

Repeat (i) and (ii) when $\sigma^2$ is unknown and $\sigma^{-2}$ has the gamma distribution with shape parameter $\alpha$ and scale parameter $\gamma$ , where $\alpha$ and $\gamma$ are known.

(iv)

In part (iii), obtain Bayes estimators of $\sigma^2$ and $l^\tau \beta /\sigma$ under the squared error loss and show that they are consistent under the condition in (ii).

Remark.

?

Corresponding exercise in Shao (2003): 4.36.

Bayesian linear regression

References