Section 10e

10.6 Detecting and Dealing with Outliers: 10.6.1 Types of Outliers, 10.6.2 Identifying High-Leverage Points, 10.6.3 Leave-One-Out Case Diagnostics, 10.6.4 Test for Outliers, 10.6.5 Other Methods

[5/100%]

Show

LaTeX

Show LaTeX

Problem 10e.1

Consider a linear function $\mathbf{d}^{T} \mathbf{\beta }$ of $\mathbf{\beta }$ . Show that the change in the estimate $\mathbf{d}^{T} \widehat{\mathbf{\beta }}$ when the $i$ th observation is deleted is

\mathbf{d}^{T} \widehat{\mathbf{\beta }}(i)-\mathbf{d}^{T} \widehat{\mathbf{\beta }} = \left(\mathbf{C}^{T} \mathbf{d}\right)_{i} e_{i} /\left(1-h_{i}\right),

where $\mathbf{C}$ is the catcher matrix $\left(\mathbf{X}^{T} \mathbf{X}\right)^{-1} \mathbf{X}^{T}$ .

Problem 10e.2

Show that

\mathbf{Y}_{D}-\mathbf{X}_{D} \widehat{\mathbf{\beta }}(D) = \left(\mathbf{I}-\mathbf{H}_{D}\right)^{-1}\left[\mathbf{Y}_{D}-\mathbf{X}_{D} \widehat{\mathbf{\beta }}(D)\right]

Problem 10e.3

Suppose that we delete a set $D$ of observations. Show that the "delete $d^{\prime \prime }$ version

\frac{\operatorname {det}\left(\mathbf{X}_{A}(D)^{T} \mathbf{X}_{A}(D)\right)}{\operatorname {det}\left(\mathbf{X}_{A} \mathbf{X}_{A}\right)}

of the Andrews-Pregibon statistic can be written as

\operatorname {det}\left(\mathbf{I}_{d}-\mathbf{H}_{D}\right)\left[1-\frac{\mathbf{e}_{D}^{T}\left(\mathbf{I}_{d}-\mathbf{H}_{D}\right)^{-1} \mathbf{e}_{D}}{\operatorname {RSS}}\right]

Problem 10e.4

Let $h_{i, A}$ be the $i$ th diagonal element of the hat matrix based on ( $\mathbf{X}, \mathbf{Y}$ ) rather than $\mathbf{X}$ . Show that $A P(i) = 1-h_{i, A}$ . Hint: Use the results of Section 10.6.3.

Problem 10e.5

(a)(b)(c)

The influence curve of the least squares estimate $\widehat{\mathbf{\beta }}$ was derived in Example 3.22. A sample version of the IC (cf. Section 3.13.3) can be defined by taking $\mathrm{z}_{0} = \left(\mathbf{x}_{i}, Y_{i}\right), t = -1 /(n-1), F$ as the e.d.f. $\widehat{F}_{n}$ , and by considering the difference $\left[T\left(F_{t}\right)-T(F)\right] / t$ rather than the derivative. This leads to the sample influence curve $\mathrm{SIC}_{i}$ , given by

\mathrm{SIC}_{i} = -(n-1)\left\{ T\left[(n-1)^{-1}\left(n \widehat{F}_{n}-\delta _{i}\right)\right]-T\left(\widehat{F}_{n}\right)\right\}

where $\delta_{i}$ puts mass 1 at ( $\mathbf{x}_{i}, Y_{i}$ ).

(a)

Show that $(n-1)^{-1}\left(n \widehat{F}_{n}-\delta_{i}\right)$ is the empirical distribution function calculated from the remaining $(n-1)$ points when the $i$ th observation has been deleted from the sample.

(b)

Hence show that for the least squares estimate functional $T$ ,

\mathrm{SIC}_{i} = (n-1)(\widehat{\mathbf{\beta }}(i)-\widehat{\mathbf{\beta }})

(c)

The quantity $\mathrm{SIC}_{i}$ is a vector. We can obtain an overall scalar measure by considering the quantity

D_{i}(\mathbf{M}, c) = c^{-1}\left(\mathrm{SIC}_{i}\right)^{T} \mathbf{M}\left(\mathbf{S I C}_{i}\right)

where $\mathbf{M}$ is a positive-definite matrix and $c$ is a positive constant. Show that if we choose $\mathbf{M} = \mathbf{X}^{T} \mathbf{X}$ and $c = p S^{2} /(n-1)^{2}$ , we obtain Cook's D.

For more details on the influence function approach to outlier detection, see Cook and Weisberg [1982, Chapter 3] and Chatterjee and Hadi [1988, Chapter 5].