Weak Law of Large Numbers

Definition

The Weak Law of Large Numbers (WLLN) states that, for a sequence of independent and identically distributed (i.i.d.) random variables $X_1,X_2,X_3,…,X_n$​ with a finite mean $\mu$ and variance $\sigma^2$, the sample average

$$\bar{X}_n=\frac{1}{n}(X_1+X_2+…+X_n)$$

converges in probability towards the expected value $\mu$ as $n$ approaches $\infty$.

In mathematical terms, for any positive number $\epsilon$,

$$P(|\bar{X}_n - \mu|<\epsilon)\rightarrow 1\text{ as } n \rightarrow \infty.\tag{1}$$

This can also be written as

$$\lim_{n\to\infty} \bar{X}_n\xrightarrow{p}\mu$$

where $\xrightarrow{p}$ denotes convergence in probability.

It's important to note that, for any finite sample size, there's still a non-zero probability that the sample mean will significantly differ from the population mean. It does not guarantee that the sample mean will equal the population mean, only that it will get arbitrarily close to it as the sample size increases.

Another simplified notation of $(1)$ is

$$\text{plim }\bar{X}_n = \mu.$$

Proof

Using Chebyshev's Inequality of the Sample mean, we know that

$$P(|\bar{X}_n - \mathbb{E}[{X}]|\geq m)\leq \frac{\sigma^2}{m^2n}.$$

Since we are dealing with probability, above expression can be written as

$$0\leq P(|\bar{X}_n - \mathbb{E}[{X}]|\geq m)\leq \frac{\sigma^2}{m^2n}.$$

We know that

$$\lim_{n\to \infty}\frac{\sigma^2}{m^2n}=0$$

as $\sigma^2$ and $m^2$ are constants, therefore

$$\begin{align*} \lim_{n\to \infty}P(|\bar{X}_n - \mathbb{E}[{X}]|&\geq m)=0,\\ \implies \lim_{n\to \infty}1-P(|\bar{X}_n - \mathbb{E}[{X}]|&< m)=0,\\ \implies \lim_{n\to \infty}P(|\bar{X}_n - \mathbb{E}[{X}]|&< m)=1. \hspace{20px}\blacksquare\\ \end{align*}$$