Martingales from harmonic and excessive functions of Markov processes

Let $\left\{ X_{n}\right\}_{n \geq 0}$ be a Markov chain with state space $\mathbb {S} = \left\{ 1,2,3, \ldots \right\}$, and transition probability matrix $P = \left(\left(p_{i j}\right)\right)$. That is, for each $n \geq 1$,

for all $i_{0}, i_{1}, i_{2}, \ldots , i_{n} \in \mathbb {S}$. Let $h: \mathbb {S} \rightarrow \mathbb {R}$ and $\rho \in \mathbb {R}$ be such that

and

Suppose $\left(X_{t}, \mathbb {P}[x]\right)$ is a Markov process with transition probabilities $P_{t}, f$ is a bounded Borel measurable function, $t_{0}>0$, and we define $M_{t}=P_{t_{0}-t} f\left(X_{t}\right)$ for $t \leq t_{0}$. Show that $\left(M_{t}, t \leq t_{0}\right)$ is a $\mathbb {P}[x]$-martingale for each $x$.

Let $f$ be an excessive function for a strong Markov process $\left(X_{t}, \mathbb {P}[x]\right)$. Let $\varepsilon >0$ and $S_{1}=\inf \left\{ t : \left|f\left(X_{t}\right)-f\left(X_{0}\right)\right| \geq \varepsilon \right\}$. Let $S_{i+1}=S_{i}+S_{1} \circ \theta_{S_{i}}$. Prove that $f\left(X_{S_{i}}\right)$ is a supermartingale with respect to the $\sigma$-fields $\mathcal{F}_{S_{i}}$ and with respect to $\mathbb {P}[x]$ for each $x$.

Suppose that $\left(Y_{n}\right)$ is a finite-state Markov chain with transition matrix $\left[p_{i j}\right]$. Suppose that $\sum_{j} p_{i j} x(j)=\lambda x(i)$ for all $i$ (the $x(i)$ are the components of a right eigenvector of the transition matrix). Put $X_{n}=\lambda^{-n} x\left(Y_{n}\right)$ and show that $\left\{ X_{n}\right\}$ is a martingale.

(see [12; p. 166]). Let $W = \left\{ W_{t}, t \geqslant 0\right\}$ be an $m$-dimensional Brownian motion and $f(x)$ a continuous superharmonic function in $\mathbb {R}^{m}$, i.e., $f(x)$, for any $x \in \mathbb {R}^{m}$, is not less than the integral of this function over any sphere centered at $x$. Prove that the process $X = \left\{ X_{t} = f\left(W_{t}\right), t \geqslant 0\right\}$ is a supermartingale with respect to the natural filtration of the Brownian motion if $\mathbb {E}\left[\left|X_{t}\right|\right] < \infty$ for all $t \geqslant 0$.

Let $X$ be a discrete-time Markov chain with countable state space $S$ and transition matrix $\mathbf{P}$. Suppose that $\psi : S \rightarrow \mathbb {R}$ is bounded and satisfies $\sum_{j \in S} p_{i j} \psi (j) \leq \lambda \psi (i)$ for some $\lambda > 0$ and all $i \in S$. Show that $\lambda^{-n} \psi \left(X_{n}\right)$ constitutes a supermartingale.

Let $X$ be a continuous-time Markov chain with finite state space $S$ and generator $\mathbf{G}$. Let $\mathbf{\eta } = \left\{ \eta (i): i \in S\right\}$ be a root of the equation $\mathbf{G} \mathbf{\eta }^{\prime } = \mathbf{0}$. Show that $\eta (X(t))$ constitutes a martingale with respect to $\mathcal{F}_{t} = \sigma (\left\{ X(u): u \leq t\right\} )$.