Law of large numbers

The sample average converges in probability towards the expected value:

\[\bar X_n \overset{P}{\to} \mu\]

That is, for any positive number $\epsilon$:

\[\lim_{n \to \infty} \mathbb{P}(|\bar X_n - \mu| < \epsilon) = 1\]

Central limit theorem

Let $(X_n)_{n \ge 1}$ be a real and independent sequence with same law such that $\mu = \mathbb{E}[X_1]$ and $\mathbb{V}[X_1]=\sigma^2$ are defined
($\mathbb{V}[X_1] \leq +\infty$). Noting $\bar{X}_n=\frac{1}{n}(X_1 + … + X_n)$, we have:

\[\sqrt{n}\frac{(\bar{X}_n-\mu)}{\sigma} \sim_{n \to \infty} \mathcal{N}(0,1)\]

Conceptual definition: we draw several samples of size $n$ from any random distribution. We compute the mean of each sample.

=> when plotting the sequence of means we get a normal distribution.

Note: the normality is satisfied from $n \geq 30$

Relationship between both theorems

The LLN states that the sample average converges toward the expected value. The CLT describes the distributional form of the fluctuations during this convergence.