Math Problems Vault

Problem 6.10.1

(a)(b)(c)

Let $N < \infty$ , and let $\mathcal{Q}$ be the space of $N \times N$ matrices with real entries, with norm

|Q| = \sup _{\mathbf{x} \neq \mathbf{0}} \frac{|Q \mathbf{x}|}{|\mathbf{x}|}, \quad Q \in \mathcal{Q}

where $\left|\mathbf{y}\right|$ is the Euclidean norm of the vector $\mathbf{y}$ , and the supremum is over all non-zero column vectors.

?

(a)

Show for $Q_{1}, Q_{2} \in \mathcal{Q}$ that

\left|Q_{1}+Q_{2}\right| \leq \left|Q_{1}\right|+\left|Q_{2}\right|, \quad \left|Q_{1} Q_{2}\right| \leq \left|Q_{1}\right| \cdot \left|Q_{2}\right|

(b)

Show for $Q \in \mathcal{Q}$ that $E_{n} := \sum_{k = 0}^{n} Q^{k} / k!$ converges with respect to $\left|\cdot \right|$ to a limit which we denote $E = E(Q)$ .

(c)

Show that $E(Q_{1}+Q_{2}) = E(Q_{1}) E(Q_{2})$ if $Q_{1}$ and $Q_{2}$ are commuting elements of $\mathcal{Q}$ .

Problem 6.10.2

Let $Y$ be an irreducible discrete-time Markov chain on a countably infinite state space $S$ , having transition matrix $\mathbf{Y} = \left(y_{i j}\right)$ satisfying $y_{i i} = 0$ for all states $i$ , and with stationary distribution $\mathbf{\nu }$ . Construct a continuous-time process $X$ on $S$ for which $Y$ is the jump chain, such that $X$ has no stationary distribution.

?

Problem 6.10.3

Let $X = (X(t): t \geq 0)$ be a Markov chain on $\mathbb {Z}$ with generator $\mathbf{G} = \left(g_{i j}\right)$ given by

g_{i, i-1} = i^{2}+1, \quad g_{i, i} = -2\left(i^{2}+1\right), \quad g_{i, i+1} = i^{2}+1, \quad i \in \mathbb {Z}

Show that $X$ is recurrent. Is $X$ positive recurrent?

?

Problem 6.10.4

Let $X$ be a Markov chain on $\mathbb {Z}$ with generator $G = \left(g_{i j}\right)$ satisfying

g_{i, i-1} = 3^{\left|i\right|}, \quad g_{i, i} = -3^{\left|i\right|+1}, \quad g_{i, i+1} = 2 \cdot 3^{\left|i\right|}, \quad i \in \mathbb {Z}

Show that $X$ is transient, but has an invariant distribution. Explain.

?

Problem 6.10.5

Let $X = \left(X_{t}: t \geq 0\right)$ be a Markov chain with generator $\mathbf{G} = \left(g_{i j}\right)$ on the finite state space $S$ , and let $f: S \rightarrow \mathbb {R}$ be a function, which we identify with the vector $f = (f(i): i \in S)$ . Show that

\mathbf{G} f(i) = \sum _{j \in S} g_{i j}(f(j)-f(i))

where $\mathbf{G} f$ denotes the standard matrix multiplication. Show that

\mathbf{G} f(i) = \lim _{t \rightarrow 0} \frac{1}{t}\left[\mathbb {E}_{i}\left[f\left(X_{t}\right)\right]-f(i)\right], \quad i \in S

and deduce that

\mathbb {E}_{i}\left[f\left(X_{t}\right)\right] = f(i)+\int _{0}^{t} \mathbb {E}_{i}\left[\mathbf{G} f\left(X_{s}\right)\right] d s

?