Probability Theory

Let $\Omega$ be a set, $\mathcal F$ be $\sigma$-field on $\Omega$, and $C\in \mathcal F$. Show that $\mathcal F_C=\left\{ C\cap A:A\in \mathcal F\right\}$ is a $\sigma$-field on $C$.

Let $\nu$ and $\lambda$ be two measures on a $\sigma$-field $\mathcal F$ on $\Omega$ such that $\nu \left(A\right)=\lambda \left(A\right)$ for any $A\in \mathcal C$, where $\mathcal C\subset \mathcal F$ is a collection having the property that if $A$ and $B$ are in $\mathcal C$, then so is $A\cap B$. Assume that there are $A_i\in \mathcal C$, $i=1,2,\ldots$, such that $\bigcup A_i=\Omega$ and $\nu \left(A_i\right)<\infty$ for all $i$. Show that $\nu \left(A\right)=\lambda \left(A\right)$ for any $A\in \sigma (\mathcal C)$, where $\sigma (\mathcal C)$ is the smallest $\sigma$-field containing $\mathcal C$.

Show that a real-valued function $f$ on a set $\Omega$ is Borel with respect to a $\sigma$-field $\mathcal F$ on $\Omega$ if and only if $f^{-1}(a,\infty )\in \mathcal F$ for all $a\in \mathbb {R}$.

Let $f$ and $g$ be real-valued functions on $\Omega$. Show that if $f$ and $g$ are Borel with respect to a $\sigma$-field $\mathcal F$ on $\Omega$, then so are $fg$, $f/g$ (when $g\ne 0$), and $af+bg$, where $a$ and $b$ are real numbers.

Let $f_i$, $i=1,2,\ldots$, be Borel functions on $\Omega$ with respect to a $\sigma$-field $\mathcal F$. Show that $\sup_n f_n$, $\inf_n f_n$, $\limsup_n f_n$, and $\liminf_n f_n$ are Borel with respect to $\mathcal F$. Also, show that the set

is in $\mathcal F$ and the function

is Borel with respect to $\mathcal F$.

Let $f$ be a Borel function on $\mathbb {R}^{2}$. Define a function $g$ from $\mathbb {R}$ to $\mathbb {R}$ as $g(x)=f(x,y_0)$, where $y_0$ is a fixed point in $\mathbb {R}$. Show that $g$ is Borel. Is it true that $f$ is Borel from $\mathbb {R}^{2}$ to $\mathbb {R}$ if $f(x,y)$ with any fixed $y$ or fixed $x$ is Borel from $\mathbb {R}$ to $\mathbb {R}$?

Let $\Omega =\left\{ \omega_i:i=1,2,\ldots \right\}$ be a countable set, $\mathcal F$ be all subsets of $\Omega$, and $\nu$ be the counting measure on $\Omega$ (i.e., $\nu \left(A\right)=$ the number of elements in $A$ for any $A\subset \Omega$). For any Borel function $f$, the integral of $f$ w.r.t. $\nu$ (if it exists) is

Let $\nu$ be a measure on a $\sigma$-field $\mathcal F$ on $\Omega$ and $f$ and $g$ be Borel functions with respect to $\mathcal F$. Show that

Let $F$ be a cumulative distribution function on the real line $\mathbb {R}$ and $a\in \mathbb {R}$. Show that

Let $F$ and $G$ be two cumulative distribution functions on the real line. Show that if $F$ and $G$ have no common points of discontinuity in the interval $[a,b]$, then

Let $Y$ be a random variable and $m$ be a median of $Y$, i.e., $\mathbb {P}\left(Y\leq m\right)\geq 1/2$ and $\mathbb {P}\left(Y\geq m\right)\geq 1/2$. Show that, for any real numbers $a$ and $b$ such that $m\leq a\leq b$ or $m\geq a\geq b$, $\mathbb {E}\left[\left|Y-a\right|\right]\leq \mathbb {E}\left[\left|Y-b\right|\right]$.

Let $X$ and $Y$ be independent random variables satisfying $\mathbb {E}\left[\left|X+Y\right|^a\right]<\infty$ for some $a>0$. Show that $\mathbb {E}\left[\left|X\right|^a\right]<\infty$.

Let $\nu$ be a $\sigma$-finite measure on a $\sigma$-field $\mathcal F$ on $\Omega$, $\lambda$ be another measure with $\lambda \ll \nu$, and $f$ be a nonnegative Borel function on $\Omega$. Show that

where $\frac{d\lambda }{d\nu }$ is the Radon--Nikodym derivative.

Let $\mathcal F_i$ be a $\sigma$-field on $\Omega_i$, $\nu_i$ be a $\sigma$-finite measure on $\mathcal F_i$, and $\lambda_i$ be a measure on $\mathcal F_i$ with $\lambda_i\ll \nu_i$, $i=1,2$. Show that $\lambda_1\times \lambda_2\ll \nu_1\times \nu_2$ and

where $\nu_1\times \nu_2$ (or $\lambda_1\times \lambda_2$) denotes the product measure of $\nu_1$ and $\nu_2$ (or $\lambda_1$ and $\lambda_2$).

Let $\mathbb {P}$ and $\mathbb {Q}$ be two probability measures on a $\sigma$-field $\mathcal F$. Assume that $f=\frac{d\mathbb {P}}{d\nu }$ and $g=\frac{d\mathbb {Q}}{d\nu }$ exists for a measure $\nu$ on $\mathcal F$. Show that

Let $F_i$ be a cumulative distribution function on the real line having a Lebesgue density $f_i$, $i=1,2$. Assume that there is a real number $c$ such that $F_1(c)<F_2(c)$. Define

Show that the probability measure $\mathbb {P}$ corresponding to $F$ satisfies $\mathbb {P}\ll m+\delta_c$, where $m$ is the Lebesgue measure and $\delta_c$ is the point mass at $c$, and find the probability density of $F$ with respect to $m+\delta_c$.

Let $X_1$ and $X_2$ be independent random variables having the standard normal distribution. Obtain the joint Lebesgue density of $(Y_1,Y_2)$, where $Y_1=\sqrt{X_1^2+X_2^2}$ and $Y_2=X_1/X_2$. Are $Y_1$ and $Y_2$ independent?

Let $X_i$, $i=1,2,3$, be independent random variables having the same Lebesgue density $f(x)=e^{-x}\; \mathbb {1}_{(0,\infty )}(x)$. Obtain the joint Lebesgue density of $(Y_1,Y_2,Y_3)$, where $Y_1=X_1+X_2+X_3$, $Y_2=X_1/(X_1+X_2)$, and $Y_3=(X_1+X_2)/(X_1+X_2+X_3)$. Are $Y_i$'s independent?

Let $X$ and $Y$ be independent random variables with cumulative distribution functions $F_X$ and $F_Y$, respectively. Show that

Show that a random variable $X$ is independent of itself if and only if $X$ is constant a.s. Can $X$ and $f(X)$ be independent, where $f$ is a Borel function?

Let $(X,Y,Z)$ be a random $3$-vector with the following Lebesgue density:

Show that $X,Y,Z$ are pairwise independent, but not independent.

Let $X$ be a random $n$-vector having the multivariate normal distribution $\mathscr {N}\left(\mu , I_n\right)$.

Let $X$ be a random variable having a cumulative distribution function $F$. Show that if $\mathbb {E}\left[X\right]$ exists, then

Let $X$ and $Y$ be random variables having the bivariate normal distribution with $\mathbb {E}\left[X\right]=\mathbb {E}\left[Y\right]=0$, $\operatorname {Var}\left[X\right]=\operatorname {Var}\left[Y\right]=1$, and $\operatorname {Cov}\left[ X, Y \right]=\rho$. Show that $\mathbb {E}\left[\max \left\{ X,Y\right\} \right]=\sqrt{(1-\rho )/\pi }$.

Let $X$ be a random variable with $\mathbb {E}\left[X^2\right]<\infty$ and let $Y=\left|X\right|$. Suppose that $X$ has a Lebesgue density symmetric about $0$. Show that $X$ and $Y$ are uncorrelated, but they are not independent.

Let $(X,Y)$ be a random $2$-vector with the following Lebesgue density:

Show that $X$ and $Y$ are uncorrelated, but they are not independent.

Let $Y$ be a random variable having the noncentral chi-square distribution $\chi_k^2(\delta )$, where $k$ is a positive integer. Show that

Let $U_1$ and $U_2$ be independent random variables having the $\chi_{n_1}^2(\delta )$ and $\chi_{n_2}^2$ distributions, respectively, and let $F=(U_1/n_1)/(U_2/n_2)$. Show that

Let $\phi_n$ be the characteristic function of a probability measure $\mathbb {P}_n$, $n=1,2,\ldots$. Let $\left\{ a_n\right\}$ be a sequence of nonnegative numbers with $\sum_{n=1}^\infty a_n=1$. Show that $\sum_{n=1}^\infty a_n\phi_n$ is a characteristic function and find its corresponding probability measure.

Find an example of two random variables $X$ and $Y$ such that $X$ and $Y$ are not independent but their characteristic functions $\phi_X$ and $\phi_Y$ satisfy $\phi_X(t)\phi_Y(t)=\phi_{X+Y}(t)$ for all $t\in \mathbb {R}$.

Let $X$ be a random variable whose characteristic function $\phi$ satisfies $\int \left|\phi (t)\right|\, dt<\infty$. Show that $(2\pi )^{-1}\int e^{-\sqrt{-1}xt}\phi (t)\, dt$ is the Lebesgue density of $X$.

Let $\phi$ be a characteristic function and $G$ be a cumulative distribution function on the real line. Show that $\int \phi (ut)\, dG(u)$ is a characteristic function on the real line.

Let $X$ and $Y$ be independent random variables. Show that if $X$ and $X-Y$ are independent, then $X$ must be degenerate.

Let $\mathbb {P}_Y$ be a discrete distribution on $\left\{ 0,1,2,\ldots \right\}$ and given $Y=y$, the conditional distribution of $X$ be the binomial distribution with size $y$ and probability $p$. Show that

Let $X$ and $Y$ be integrable random variables on the probability space $(\Omega ,\mathcal F,\mathbb {P})$ and $\mathcal A$ be a sub-$\sigma$-field of $\mathcal F$. Show that

Let $X$ be an integrable random variable on the probability space $(\Omega ,\mathcal F,\mathbb {P})$ and $\mathcal A$ and $\mathcal A_0$ be $\sigma$-fields satisfying $\mathcal A_0\subset \mathcal A\subset \mathcal F$. Show that $\mathbb {E}\left[ \mathbb {E}\left[X\right][\mathcal A] \mid \mathcal A_0 \right]=\mathbb {E}\left[ X \mid \mathcal A_0 \right]=\mathbb {E}\left[ \mathbb {E}\left[X\right][\mathcal A_0] \mid \mathcal A \right]$ a.s.

Let $X$ be an integrable random variable on the probability space $(\Omega ,\mathcal F,\mathbb {P})$, $\mathcal A$ be a sub-$\sigma$-field of $\mathcal F$, and $Y$ be another random variable satisfying $\sigma (Y)\subset \mathcal A$ and $\mathbb {E}\left[\left|XY\right|\right]<\infty$. Show that

Let $X_1,X_2,\ldots$ and $X$ be integrable random variables on the probability space $(\Omega ,\mathcal F,\mathbb {P})$. Assume that $0\leq X_1\leq X_2\leq \cdots \leq X$ and $\lim_nX_n=X$ a.s. Show that for any $\sigma$-field $\mathcal A\subset \mathcal F$,

Let $X_1,X_2,\ldots$ be integrable random variables on the probability space $(\Omega ,\mathcal F,\mathbb {P})$. Show that for any $\sigma$-field $\mathcal A\subset \mathcal F$,

Let $X$ and $Y$ be integrable random variables on the probability space $(\Omega ,\mathcal F,\mathbb {P})$ and $\mathcal A\subset \mathcal F$ be a $\sigma$-field. Show that $\mathbb {E}\left[Y\mathbb {E}\left[ X \mid \mathcal A \right]\right]=\mathbb {E}\left[X\mathbb {E}\left[ Y \mid \mathcal A \right]\right]$, assuming that both integrals exist.

Let $X,X_1,X_2,\ldots$ be a sequence of integrable random variables on the probability space $(\Omega ,\mathcal F,\mathbb {P})$ and $\mathcal A\subset \mathcal F$ be a $\sigma$-field. Suppose that $\lim_n\mathbb {E}\left[X_nY\right]=\mathbb {E}\left[XY\right]$ for every integrable (or bounded) random variable $Y$. Show that $\lim_n\mathbb {E}\left[\mathbb {E}\left[ X_n \mid \mathcal A \right]Y\right]=\mathbb {E}\left[\mathbb {E}\left[ X \mid \mathcal A \right]Y\right]$ for every integrable (or bounded) random variable $Y$.

Let $X$ be a nonnegative integrable random variable on the probability space $(\Omega ,\mathcal F,\mathbb {P})$ and $\mathcal A\subset \mathcal F$ be a $\sigma$-field. Show that

Let $X$ and $Y$ be independent integrable random variables on a probability space and $f$ be a nonnegative convex function. Show that $\mathbb {E}\left[f(X+Y)\right]\geq \mathbb {E}\left[f(X+\mathbb {E}\left[Y\right])\right]$.

Let $X$ be an integrable random variable with a Lebesgue density $f$ and let $Y=g(X)$, where $g$ is a function with positive derivative on $(0,\infty )$ and $g(x)=g(-x)$. Find an expression for $\mathbb {E}\left[ X \mid Y \right]$ and verify that it is indeed the conditional expectation.

Let $X$, $Y$, and $Z$ be random variables on a probability space. Suppose that $\mathbb {E}\left[\left|X\right|\right]<\infty$ and $Y=h(Z)$ with a Borel $h$. Show that

Let $X_1,X_2,\ldots$ be a sequence of identically distributed random variables with $\mathbb {E}\left[\left|X_1\right|\right]<\infty$ and let

. Show that $\lim_n\mathbb {E}\left[Y_n\right]=0$ and $\lim_nY_n=0$ a.s.

Let $X,X_1,X_2,\ldots$ be random variables. Find an example for each of the following cases:

Let $X_n$ be a random variable and $m_n$ be a median of $X_n$, $n=1,2,\ldots$. Show that if $X_n\to_d X$ for a random variable $X$, then any limit point of $m_n$ is a median of $X$.

Show that if $X_n\to_d X$ and $X=c$ a.s. for a real number $c$, then $X_n\to_p X$.

Let $X_1,X_2,\ldots$ be random variables. Show that $\left\{ \left|X_n\right|\right\}$ is uniformly integrable if one of the following condition holds:

1. $\sup_n\mathbb {E}\left[\left|X_n\right|^{1+\delta }\right]<\infty$ for a $\delta >0$;

2. $\mathbb {P}\left(\left|X_n\right|\geq c\right)\leq \mathbb {P}\left(\left|X\right|\geq c\right)$ for all $n$ and $c>0$, where $X$ is an integrable random variable.

Let $\left\{ X_n\right\}$ and $\left\{ Y_n\right\}$ be sequences of random variables such that $X_n$ diverges to $\infty$ in probability and $Y_n$ is bounded in probability. Show that $X_n+Y_n$ diverges to $\infty$ in probability.

Let $X,X_1,X_2,\ldots$ be random variables. Show that if $\lim_nX_n=X$ a.s., then $\sup_{m\geq n}\left|X_m\right|$ is bounded in probability.

Let $\left\{ X_n\right\}$ and $\left\{ Y_n\right\}$ be two sequences of random variables such that $X_n$ is bounded in probability and, for any real number $t$ and $\epsilon >0$, $\lim_n\left[\mathbb {P}\left(X_n\leq t,Y_n\geq t+\epsilon \right)+\mathbb {P}\left(X_n\geq t+\epsilon ,Y_n\leq t\right)\right]=0$. Show that $X_n-Y_n\to_p0$.

Let $F_n$, $n=0,1,2,\ldots$, be cumulative distribution functions such that $F_n\to F_0$ for every continuity point of $F_0$. Let $U$ be a random variable having the uniform distribution on the interval $[0,1]$ and let $G_n(U)=\sup \left\{ x:F_n(x)\leq U\right\}$, $n=0,1,2,\ldots$. Show that $G_n(U)\to_pG_0(U)$.

Let $\left\{ X_n\right\}$ be a sequence of independent and identically distributed random variables. Show that there does not exist a sequence of real numbers $\left\{ c_n\right\}$ such that $\lim_n\sum_{i=1}^n(X_i-c_i)$ exists a.s., unless the distribution of $X_1$ is degenerate.

Let $\mathbb {P},\mathbb {P}_1,\mathbb {P}_2,\ldots$ be probability measures such that $\lim_n\mathbb {P}_{n}\left(O\right)=\mathbb {P}\left(O\right)$ for any open set $O$ with $\mathbb {P}\left(\partial O\right)=0$, where $\partial A$ is the boundary of the set $A$. Show that $\lim_n\mathbb {P}_{n}\left(A\right)=\mathbb {P}\left(A\right)$ for any Borel $A$ with $\mathbb {P}\left(\partial A\right)=0$.