Large Sample Distribution Theory

Convergence in Probability

Definition

The random variable $X_n$ converges in probability to a constant $c$ if

$$\lim_{n\to \infty}\text{ Prob }(|X_n-c|>\varepsilon)=0$$

for any positive $\varepsilon$.

If $X_n$ converges in probability to $c$, then we write

$$\text{plim }X_n=c.$$

Another way to write is the following

$$\lim_{n\to\infty} {X}_n\xrightarrow{p}c,$$

or

$${X}_n\xrightarrow{p}c.$$

Example

Let $X_n \sim \text{ Exponential}(n)$, show that ${X}_n\xrightarrow{p}0$.

Hint: CDF of $X_n \sim \text{ Exponential}(\lambda)$ id given as

$$F(x;\lambda)= \begin{cases} 1- e^{-\lambda x} & x\geq 0, \\ 0 &x< 0 \\ \end{cases}$$

Solution:

Using the definition of convergence in probability, we have

$$\begin{align*} \lim_{n\to \infty}\text{Prob}(|X_n-0|>\varepsilon) &= \lim_{n\to \infty}\text{Prob}(X_n>\varepsilon) &\hspace{50px}\text{(since } X_n\geq0)\\ &= \lim_{n\to \infty}e^{-n\varepsilon} &\hspace{50px}\text{(since } X_n \sim \text{ Exponential}(n))\\ &=0, \forall \varepsilon>0. \end{align*}$$

Convergence in Distribution

Definition

A sequence of random variables $X_1, X_2, X_3,\cdots$ converges in distribution to a random variable $X$, shown by $X_n\xrightarrow{d}X$, if,

$$\lim_{n\to \infty}F_{X_n}(x)=F_X(x),$$

for all $x$ at which $F_X(x)$ is continuous.

Limiting Distribution

If $X_n$ converges in distribution to $X$, where $F_{X_n}(x)$ is the CDF of $X_n$, then $F_X(x)$ is the limiting distribution of $X_n$.

Example

Let $X_1,X_2, X_3,\cdots$ be a sequence of random variable such that

$$F_{X_n}(x)= \begin{cases} 1-\Big(1-\frac{1}{n}\Big)^{nx} & x\geq 0 \\ 0 & \text{otherwise} \\ \end{cases}$$

Show that $X_n$ converges in distribution to $\text{Exponential(1)}$.

Solution:

Let $X∼\text{Exponential(1)}$.
For $x\leq0$, we have

$$F_{X_n}(x)=F_X(x)=0, \hspace{15px} \text{ for n}=2,3,4,\cdots.$$

For $x\geq0$, we have

$$\begin{align*} \lim_{n\to \infty}F_{X_n}(x)&=\lim_{n\to \infty}\Big(1−\Big(1−\frac{1}{n}\Big)^{nx}\Big)\\ &=1−\lim_{n\to\infty}\Big(1−\frac{1}{n}\Big)^{nx}\\ &=1−e^{−x}\\ &=F_X(x), \forall x. \end{align*}$$

Thus, we conclude that $X_n \xrightarrow{d}X$.