Random processes

Let $\left\{ X_{n}\right\}$ be a Markov chain on the state space $S = \left\{ 0,1\right\}$ with transition matrix

where $\alpha +\beta > 0$. Find:

Let $\left\{ N(t): t \geq 0\right\}$ be a Poisson process of intensity $\lambda$, and let $T_{0}$ be an independent random variable such that $\mathbb {P}\left(T_{0} = \pm 1\right) = \frac{1}{2}$. Define $T(t) = T_{0}(-1)^{N(t)}$. Show that $\left\{ T(t): t \geq 0\right\}$ is stationary and find:

An integer-valued counting process $\left\{ N(t): t \geq 0\right\}$ with $N(0) =$ 0 is called crudely stationary if $p_{k}(s, t) = \mathbb {P}\left(N(s+t)-N(s) = k\right)$ depends only on the length $t-s$ and not on the location $s$. It is called simple if, almost surely, it has jump discontinuities of size 1 only. Show that, for a simple crudely stationary process $N, \lim_{t \downarrow 0} t^{-1} \mathbb {P}\left(N(t) > 0\right) = \mathbb {E}\left[N(1)\right]$.

Let $X = \left\{ X_{n}: n \geq 0\right\}$ be an ergodic Markov chain started in its stationary distribution $\pi$. Let $A$ be a subset of states, with stationary probability $\pi (A)$, and let $T_{A} = \min \left\{ n \geq 1: X_{n} \in A\right\}$. Show that $\mathbb {E}\left[T_{A} \mid X_{0} \in A\right] = 1 / \pi (A)$.

Let $\left(f_{n}: n \geq 1\right)$ be a probability distribution on the positive integers, and define a sequence $(u_{n}: n \geq 0)$ by $u_{0} = 1$ and $u_{n} = \sum_{r = 1}^{n} f_{r} u_{n-r}, n \geq 1$. Explain why such a sequence is called a renewal sequence, and show that $u$ is a renewal sequence if and only if there exists a Markov chain $U$ and a state $s$ such that $u_{n} = \mathbb {P}\left(U_{n} = s \mid U_{0} = s\right)$.

Let $\left\{ X_{i}: i \geq 1\right\}$ be the inter-event times of a discrete renewal process on the integers. Show that the excess lifetime $B_{n}$ constitutes a Markov chain. Write down the transition probabilities of the sequence $\left\{ B_{n}\right\}$ when reversed in equilibrium. Compare these with the transition probabilities of the chain $U$ of your solution to Exercise (8.3.1).

Let $(u_{n}: n \geq 1)$ satisfy $u_{0} = 1$ and $u_{n} = \sum_{r = 1}^{n} f_{r} u_{n-r}$ for $n \geq 1$, where $\left(f_{r}: r \geq 1\right)$ is a non-negative sequence. Show that:

Events occur at the times of a discrete-time renewal process $N$ (see Example (5.2.15)). Let $u_{n}$ be the probability of an event at time $n$, with generating function $U(s)$, and let $F(s)$ be the probability generating function of a typical inter-event time. Show that, if $\left|s\right| < 1$ :

Prove Theorem (8.3.5): Poisson processes are the only renewal processes that are Markov chains.

The two tellers in a bank each take an exponentially distributed time to deal with any customer; their parameters are $\lambda$ and $\mu$ respectively. You arrive to find exactly two customers present, each occupying a teller.

Customers arrive at a desk according to a Poisson process of intensity $\lambda$. There is one clerk, and the service times are independent and exponentially distributed with parameter $\mu$. At time 0 there is exactly one customer, currently in service. Show that the probability that the next customer arrives before time $t$ and finds the clerk busy is

Vehicles pass a crossing at the instants of a Poisson process of intensity $\lambda$; you need a gap of length at least $a$ in order to cross. Let $T$ be the first time at which you could succeed in crossing to the other side. Show that $\mathbb {E}\left[T\right] = \left(e^{a \lambda }-1\right) / \lambda$, and find $\mathbb {E}\left[e^{\theta T}\right]$.

Suppose there are two lanes to cross, carrying independent Poissonian traffic with respective rates $\lambda$ and $\mu$. Find the expected time to cross in the two cases when:

Customers arrive at the instants of a Poisson process of intensity $\lambda$, and the single server has exponential service times with parameter $\mu$. An arriving customer who sees $n$ customers present (including anyone in service) will join the queue with probability $(n+1) /(n+2)$, otherwise leaving for ever. Under what condition is there a stationary distribution? Find the mean of the time spent in the queue (not including service time) by a customer who joins it when the queue is in equilibrium. What is the probability that an arrival joins the queue when in equilibrium?

Customers enter a shop at the instants of a Poisson process of rate 2. At the door, two representatives separately demonstrate a new corkscrew. This typically occupies the time of a customer and the representative for a period which is exponentially distributed with parameter $1$, independently of arrivals and other demonstrators. If both representatives are busy, customers pass directly into the shop. No customer passes a free representative without being stopped, and all customers leave by another door. If both representatives are free at time $0$, show the probability that both are busy at time $t$ is $\frac{2}{5}-\frac{2}{3} e^{-2 t}+\frac{4}{15} e^{-5 t}$.

For a Wiener process $W$ with $W(0) = 0$, show that

Calculate $\mathbb {P}\left(W(s) > 0, W(t) > 0, W(u) > 0\right)$ when $s < t < u$.

Let $W$ be a Wiener process. Show that, for $s < t < u$, the conditional distribution of $W(t)$ given $W(s)$ and $W(u)$ is normal

Deduce that the conditional correlation between $W(t)$ and $W(u)$, given $W(s)$ and $W(v)$, where $s < t < u < v$, is

For what values of $a$ and $b$ is $a W_{1}+b W_{2}$ a standard Wiener process, where $W_{1}$ and $W_{2}$ are independent standard Wiener processes?

Show that a Wiener process $W$ with variance parameter $\sigma^{2}$ satisfies

The process $W$ is said to have finite quadratic variation.

Let $W$ be a Wiener process. Which of the following define Wiener processes?

Find the distribution of $\int_{0}^{t}[W(u) / u] d u$ where $W$ is the Wiener process. [The integral is interpreted as the limit as $\epsilon \downarrow 0$ of the integral from $\epsilon$ to $t$.]

Wiener process in $n$ dimensions. An $n$-dimensional Wiener process is a process $W(t) = \left(W_{1}(t), W_{2}(t), \ldots , W_{n}(t)\right)$ taking values in $\mathbb {R}^{n}$ with independent increments, started at the origin $W(0) = 0$, and such that $W(t)-W(s)$ is multivariate normal with means 0 and covariance matrix $(t-s) \mathbf{I}$ where $\mathbf{I}$ is the $n \times n$ identity matrix. Let $W(t)$ be an $n$-dimensional Wiener process, and let $A$ be an orthonormal $n \times n$ matrix. Show that $A W(t)$ is an $n$-dimensional Wiener process.

Let $W$ be a standard Wiener process. Show that, for $p \geq 0$,

where

Prove that the characteristic function $\phi (t, \theta ) = \mathbb {E}\left[e^{i \theta X(t)}\right]$ of a Lévy process $X$ is a continuous function of $t$.

Show that a compound Poisson process is a Lévy process, and find its Lévy symbol.

Verify the formula of Example (8.6.6) for the Laplace exponent of the Moran gamma process.

Prove Theorem (8.6.7): the property of being a Lévy process is preserved under time change by an independent subordinator.

Let $N$ be a Poisson process of rate $1$, and $T$ an independent random variable with the $\Gamma (1, t)$ distribution. Show that $Y = N(T)$ has probability generating function $\mathbb {E}\left[s^{Y}\right] = (2-s)^{-t}$ for $s < 2$.

Let $X$ be a Lévy process with finite variances. Show that the following are martingales:

Let $X$ be a continuous-time martingale and $T$ an independent subordinator. Show that, subject to the moment condition $\mathbb {E}\left[\left|Y(t)\right|\right] < \infty , Y(t) = X(T(t))$ defines a martingale with respect to a suitable filtration. Show that the moment condition is valid whenever $X$ is a positive martingale.

Let $X$ be stable, and let $X_{1}, X_{2}, \ldots , X_{n}$ be a random sample distributed as $X$. Show that, for $n \geq 1$, there exist $A_{n}, B_{n}$ such that $X_{1}+X_{2}+\cdots +X_{n} \stackrel{\mathrm{D}}{ = } B_{n} X+A_{n}$.

Prove that, for a non-degenerate, self-similar process $X$ with stationary increments, finite variances, and Hurst exponent $H > 0$,

Show that a stable distribution is infinitely divisible.

Show that a self-similar Lévy process either is a Wiener process or has infinite variances.

Prove that a distribution with characteristic function $\phi (\theta ) = \exp \left(-|\theta |^{\alpha }\right)$, where $\alpha \in (0,2]$, is strictly $\alpha$-stable and symmetric.

Let $X$ and $Y$ be independent stable random variables, where $X$ is symmetric with exponent $\alpha \in (0,2]$, and $Y$ is non-negative with Laplace transform $M(\theta ) = \exp \left(-k \theta^{\beta }\right)$ for $\theta \in [0, \infty )$, where $1 \neq \beta \in (0,2]$. Show that $Z = X Y^{1 / \alpha }$ is symmetric and $\alpha \beta$-stable. Deduce that, for independent $N(0,1)$ random variables $U, V$, the ratio $Z = U / V$ has the Cauchy distribution.

Let $Z$ be a continuous-time Markov chain, and let $T$ be an independent subordinator. Show that $X(t) = Z(T(t))$ defines a Markov chain.

Let $Z = \left\{ Z_{n}: n \geq 0\right\}$ be a discrete-time Markov chain with $n$-step transition probabilities $z_{i j}(n)$. Let $N$ be a Poisson process of rate $\lambda$ that is independent of $Z$, and set $X(t) = Z_{N(t)}$. Show that $X$ is a continuous-time Markov chain with transition probabilities

Let $W$ be the standard Wiener process, and let $T$ be an independent stable subordinator with $\mathbb {E}\left[e^{-\theta T(t)}\right] = \exp \left(-t \theta^{a / 2}\right)$ where $a \in (0,2)$ and $\theta \geq 0$. Show that the time-changed process $Y(t) = W(T(t))$ is a symmetric Lévy process.

Markov, but not strong Markov. Let $V$ be exponentially distributed with parameter 1, and define $X = \left\{ X(t): t \geq 0\right\}$ by: $X(t) = 0$ for $t \leq V$, and $X(t) = t-V$ for $t \geq V$.

Let $\left\{ Z_{n}\right\}$ be a sequence of uncorrelated real-valued variables with zero means and unit variances, and define the 'moving average'

for constants $\alpha_{0}, \alpha_{1}, \ldots , \alpha_{r}$. Show that $Y$ is stationary and find its autocovariance function.

Let $\left\{ Z_{n}\right\}$ be a sequence of uncorrelated real-valued variables with zero means and unit variances. Suppose that $\left\{ Y_{n}\right\}$ is an 'autoregressive' stationary sequence in that it satisfies $Y_{n} = \alpha Y_{n-1}+Z_{n}$, $-\infty < n < \infty$, for some real $\alpha$ satisfying $\left|\alpha \right| < 1$. Show that $Y$ has autocovariance function $c(m) = \alpha^{\left|m\right|} /\left(1-\alpha^{2}\right)$.

Let $\left\{ X_{n}\right\}$ be a sequence of independent identically distributed Bernoulli variables, each taking values 0 and 1 with probabilities $1-p$ and $p$ respectively. Find the mass function of the renewal process $N(t)$ with interarrival times $\left\{ X_{n}\right\}$.

Customers arrive in a shop in the manner of a Poisson process with parameter $\lambda$. There are infinitely many servers, and each service time is exponentially distributed with parameter $\mu$. Show that the number $Q(t)$ of waiting customers at time $t$ constitutes a birth-death process. Find its stationary distribution.

Let $X(t) = Y \cos (\theta t)+Z \sin (\theta t)$ where $Y$ and $Z$ are independent $N(0,1)$ random variables, and let $\tilde{X}(t) = R \cos (\theta t+\Psi )$ where $R$ and $\Psi$ are independent. Find distributions for $R$ and $\Psi$ such that the processes $X$ and $\widetilde{X}$ have the same fdds.

Customers arrive at the entrance to a queueing system at the instants of an non-homogeneous Poisson process with rate function $\lambda (t)$. Their subsequent service histories are independent of each other, and a customer arriving at time $s$ is in state $A$ at time $s+t$ with probability $p(s, t)$. Show that the number of customers in state $A$ at time $t$ is Poisson with parameter $\int_{-\infty }^{t} \lambda (u) p(u, t-u) d u$.

An insurance company receives premiums (net of costs) at unit rate per unit time, and the claims of size $X_{1}, X_{2}, \ldots$ are independent random variables with common distribution function $F$, arriving at the instants of a Poisson process of rate 1. Show that the probability $r(y)$ of ruin (that is, assets becoming negative) for an initial capital $y$ satisfies

Let $X$ be $\mathrm{fBM}_{H}$ with $\operatorname {Var}\left[X(1)\right] = 1$, where $H \in (0,1)$. Show that:

Holtsmark distribution for stellar gravity, Problem (6.15.57) revisited. Suppose stars are distributed in $\mathbb {R}^{3}$ in the manner of a Poisson process with intensity $\lambda$, and let $G_{\lambda }$ be the $x$-coordinate of their aggregated gravitational force exerted on a unit mass at the origin.

In a Prague teashop (U Myšáka) before the Velvet Revolution of 1989, customers queue at the entrance for a blank bill. In the shop there are separate counters for coffee, sweetcakes, pretzels, milk, drinks, and ice cream, and queues form at each of these. At each service point the customers' bills are marked appropriately. There is a restricted number $N$ of seats, and departing customers have to queue in order to pay their bills. If interarrival times and service times are exponentially distributed and the process is in equilibrium, find how much longer a greedy customer must wait if he insists on sitting down. Answers on a postcard to the authors, please.