Crossings and convergence

Give a reasonable definition of a downcrossing of the interval $[a, b]$ by the random sequence $Y_{0}, Y_{1}, \ldots$.

Let $(Y, \mathscr {F})$ be a supermartingale with finite means, and let $U_{n}(a, b ; Y)$ be the number of upcrossings of the interval $[a, b]$ up to time $n$. Show that

Deduce that $\mathbb {E}\left[U_{n}(a, b ; Y)\right] \leq a /(b-a)$ if $Y$ is non-negative and $a \geq 0$.

Let $X$ be a Markov chain with countable state space $S$ and transition matrix $P$. Suppose that $X$ is irreducible and recurrent, and that $\psi : S \rightarrow S$ is a bounded function satisfying $\sum_{j \in S} p_{i j} \psi (j) \leq \psi (i)$ for $i \in S$. Show that $\psi$ is a constant function.

Let $Z_{1}, Z_{2}, \ldots$ be independent random variables such that:

where $a_{1} = 2$ and $a_{n} = 4 \sum_{j = 1}^{n-1} a_{j}$. Show that $Y_{n} = \sum_{j = 1}^{n} Z_{j}$ defines a martingale. Show that $Y = \lim Y_{n}$ exists almost surely, but that there exists no $M$ such that $\mathbb {E}\left[\left|Y_{n}\right|\right] \leq M$ for all $n$.

Let $x_{1}, x_{2}, \ldots , x_{r} \in \mathbb {R}$, and let the sequence $\left\{ X_{n}: n \geq 1\right\}$ of random variables be given as follows. We set

where $U(n)$ and $V(n)$ are uniformly distributed on $\left\{ 1,2, \ldots , n-1\right\}$, and the random variables $\left\{ U(n), V(n): n > r\right\}$ are independent.

Show that

is a martingale with respect to the sequence $\left\{ X_{n}\right\}$. Enthusiasts seeking a challenge are invited to show that the $M$ converges almost surely and in mean square to a non-degenerate limit.

Let $R_{n}$ and $B_{n}$ be the numbers of red and blue balls, respectively, in an urn at the $n$th stage, and assume $R_{0} = B_{0} = 1$. At each stage, a ball is drawn and returned together with a fresh ball of the other colour. Show that $M_{n} = \left(B_{n}-R_{n}\right)\left(B_{n}+R_{n}-1\right)$ defines a martingale. Does it converge almost surely?