Chapter 23

Optimal stopping

[10/0%]
LaTeX
Problem 23.1

Show that if ff is excessive, then 1−e−f1-e^{-f} is excessive. Thus, for some purposes it is enough to look at bounded excessive functions.

?
Problem 23.2

Show that if ff and gg are excessive, then f∧gf \wedge g is excessive.

?
Problem 23.3

Let AtA_{t} be an additive functional (defined in (22.4)) and let f(x)=Ex[A∞]f(x)=\mathbb {E}_{x}\left[A_{\infty }\right]. Show that ff is excessive.

?
Problem 23.4

Let ff be an excessive function for a strong Markov process (Xt,P[x])\left(X_{t}, \mathbb {P}[x]\right). Let ε>0\varepsilon >0 and S1=inf⁡{t:∣f(Xt)−f(X0)∣≥ε}S_{1}=\inf \left\{ t : \left|f\left(X_{t}\right)-f\left(X_{0}\right)\right| \geq \varepsilon \right\}. Let Si+1=Si+S1∘θSiS_{i+1}=S_{i}+S_{1} \circ \theta_{S_{i}}. Prove that f(XSi)f\left(X_{S_{i}}\right) is a supermartingale with respect to the σ\sigma-fields FSi\mathcal{F}_{S_{i}} and with respect to P[x]\mathbb {P}[x] for each xx.

?
Problem 23.5

For each nn, let AtnA_{t}^{n} be an additive functional with continuous paths and suppose that fn(x)=Ex[A∞n]f_{n}(x)=\mathbb {E}_{x}\left[A_{\infty }^{n}\right] is finite for every xx. Suppose AtA_{t} is a continuous additive functional with f(x)=Ex[A∞]f(x)=\mathbb {E}_{x}\left[A_{\infty }\right] also finite for each xx. Suppose fnf_{n} converges to ff uniformly. Prove that for each xx, with P[x]\mathbb {P}[x]-probability one, AtnA_{t}^{n} converges to AtA_{t}, uniformly over t≥0t \geq 0.

Hint: Use Proposition 9.11.

?
Problem 23.6

Suppose ff is bounded and excessive, λ≥0\lambda \geq 0, and A={y:f(y)≤λ}A=\left\{ y: f(y) \leq \lambda \right\}. Prove that if x∈Arx \in A^{r} (i.e., xx is regular for AA ), then f(x)≤λf(x) \leq \lambda.

Hint: Use the optional section theorem (Theorem 16.12) to find stopping times TmT_{m} whose graphs are contained in {(t,ω):t≤1/m,f(Xt)≤λ}\left\{ (t, \omega ): t \leq 1 / m, f\left(X_{t}\right) \leq \lambda \right\} with P[x]\mathbb {P}[x]-probability at least 1−(1/m)1-(1 / m).

Lemma 23.1: If ff is excessive, there exist functions gn≥0g_{n} \geq 0 such that UgnU g_{n} increases up to ff, where Ugn(x)=Ex[∫0∞gn(Xs) ds]U g_{n}(x) = \mathbb {E}_{x}\left[\int_0^\infty g_n(X_s)\, ds\right].

If the gng_{n} are as in Lemma 23.1, write

Ugn(x)=Ex[∫0Tmgn(Xs)ds]+Ex[Ugn(XTm)]≤Ex[∫0Tmgn(Xs)ds]+Ex[f(XTm)]≤Ex[∫0Tmgn(Xs)ds]+λ+∥f∥∞/m \begin{aligned} U g_{n}(x) & =\mathbb {E}_{x}\left[\int _{0}^{T_{m}} g_{n}\left(X_{s}\right) d s\right]+\mathbb {E}_{x}\left[U g_{n}\left(X_{T_{m}}\right)\right] \\ & \leq \mathbb {E}_{x}\left[\int _{0}^{T_{m}} g_{n}\left(X_{s}\right) d s\right]+\mathbb {E}_{x}\left[f\left(X_{T_{m}}\right)\right] \\ & \leq \mathbb {E}_{x}\left[\int _{0}^{T_{m}} g_{n}\left(X_{s}\right) d s\right]+\lambda +\left\| f\right\| _{\infty } / m \end{aligned}

Let m→∞m \rightarrow \infty, then n→∞n \rightarrow \infty.

?
Problem 23.7

Suppose ff is bounded and excessive, λ≥0\lambda \geq 0, and B={y:f(y)≥λ}B=\left\{ y: f(y) \geq \lambda \right\}. Prove that if x∈Brx \in B^{r}, then f(x)≤λf(x) \leq \lambda.

Hint: Use the optional stopping theorem as in Exercise 23.6 to find stopping times RmR_{m} analogous to the TmT_{m}. Write

f(x)≥Ex[f(XRm)]≥λ−∥f∥∞/m f(x) \geq \mathbb {E}_{x}\left[f\left(X_{R_{m}}\right)\right] \geq \lambda -\left\| f\right\| _{\infty } / m

and then let m→∞m \rightarrow \infty.

?
Problem 23.8
?
(1)

Suppose ff is bounded and excessive, x∈S,ε>0x \in \mathcal{S}, \varepsilon >0, and C={y:∣f(y)−f(x)∣≥ε}C=\left\{ y:\left|f(y)-f(x)\right| \geq \varepsilon \right\}. Use Exercises 23.6 and 23.7 to show that if z∈Crz \in C^{r}, then ∣f(z)−f(x)∣≥ε\left|f(z)-f(x)\right| \geq \varepsilon.

(2)

Let f,εf, \varepsilon, and xx be as in (1) and set S=inf⁡{t>0:∣f(Xt)−f(x)∣≥ε}S=\inf \left\{ t>0:\left|f\left(X_{t}\right)-f(x)\right| \geq \varepsilon \right\}. Use Exercise 20.9 to show that ∣f(XS)−f(x)∣≥ε\left|f\left(X_{S}\right)-f(x)\right| \geq \varepsilon with P[x]\mathbb {P}[x]-probability one.

(3)

Let f,ε,xf, \varepsilon , x, and SS be as in (2). Define S=0S=0 and Si+1=Si+S∘θSiS_{i+1}=S_{i}+S \circ \theta_{S_{i}}. By Exercise 23.4, f(XSi)f\left(X_{S_{i}}\right) is a positive supermartingale. Use Corollary A. 36 to show Si→∞,P[x]S_{i} \rightarrow \infty , \mathbb {P}[x]-a.s. Deduce that with P[x]\mathbb {P}[x]-probability one, f(Xt)f\left(X_{t}\right) has paths that are right continuous with left limits.

(4)

Use Exercise 23.1 to show that if ff is excessive but not necessarily bounded, then f(Xt)f\left(X_{t}\right) has paths that are right continuous with left limits.

Problem 23.9
?
(1)

Show that every continuous function is lower semicontinuous.

(2)

Show that if ff is lower semicontinuous and x∈Sx \in \mathcal{S}, then

lim inf⁡y→xf(y)≥f(x) \liminf _{y \rightarrow x} f(y) \geq f(x)
(3)

Show that if fnf_{n} is a sequence of continuous functions increasing to ff, then ff is lower semicontinuous.

Problem 23.10

Suppose gg is non-negative, bounded, and continuous, and Assumption 20.1 holds. Let g0=gg_{0}=g and define gn(x)=sup⁡t≥0Ptgn−1(x)g_{n}(x)=\sup_{t \geq 0} P_{t} g_{n-1}(x) for n≥1n \geq 1. Prove that gng_{n} increases to the least excessive majorant of gg.

?