Semi-Markov Processes

Find the (limiting) probability that the taxi's most recent stop was at location $i$, $i = 1, 2, 3$.

What is the (limiting) probability that the taxi is heading for location $2$?

What fraction of time is the taxi traveling from location $2$ to location $3$?

Assume that the Markov process in Exercise 7.5 is changed in the following way: whenever the process enters state $0$, the time spent before leaving state $0$ is now a uniformly distributed rv, taking values from $0$ to $2/\lambda$. All other transitions remain the same. For this new process, determine whether the successive epochs of entry to state $0$ form renewal epochs, whether the successive epochs of exit from state $0$ form renewal epochs, and whether the successive entries to any other given state $i$ form renewal epochs.

For each $i$, find both the time-average interval and the time-average number of overall state transitions between successive visits to $i$.

Is this modified process a Markov process in the sense that $\mathbb {P}\left\{ X(t) = i \mid X(\tau ) = j, X(s) = k\right\} = \mathbb {P}\left\{ X(t) = i \mid X(\tau ) = j\right\}$ for all $0 < s < \tau < t$ and all $i, j, k$? Explain.

Suppose a colleague has calculated the steady-state probabilities $\left\{ p_{i}\right\}$ of being in state $i$ for each $i \geq 0$. For each $i \geq 0$, find the steady-state probability $\pi$ of state $i$ in the embedded Markov chain. Give your solution as a function of $\rho$, $\pi_{i}$, and $p_{0}$.

Calculate $p_{0}$ as a function of $\rho$.

Find $\pi_{i}$ as a function of $\rho$ and $p_{i}$.

Is $p_{i}$ the same as the steady-state probability that the queueing system contains $i$ customers at a given time? Explain carefully.

Find $P_{0j}$ for $j \geq 0$.

Find $P_{ij}$ for $i > 0$; $j \geq i-1$.