Independent and Identically Distributed Gamma
This web page gives a few examples of the central limit theorem (CLT) in action.
We know the sum of gamma random variables is gamma. The CLT says when the number of terms in the sum is large, this gamma distribution should be approximately normal.
But how large the number of terms in the sum has to be depends on the actual gamma distribution chosen for the terms.
Change the assignments of alpha and n in the
first two lines to experiment.
The larger alpha is, the less skewed the distribution of
the individual terms is and the smaller n has to be to get
good normal approximation. You should be able to see this for yourself,
if you experiment.
Independent and Identically Distributed Bernoulli
We know the sum of Bernoulli random variables is binomial. The CLT says when the number of terms in the sum is large, this binomial distribution should be approximately normal. This shows there is no problem with discrete distributions converging in distribution to the (continuous) normal distribution
But how large the number of terms in the sum (the binomial sample size) has to be depends on the actual Bernoulli distribution chosen for the terms.
There is a rule of thumb
taught in many intro statistics courses
that the normal approximation is OK if
n p ≥ 5 and
n (1 − p) ≥ 5.
But, of course, this
is, strictly speaking, nonsense. There is no sharp borderline between
bad and good normal approximation. It all depends on what probabilities
you are calculating and how much accuracy you want.
Change the assignments of p and n in the
first two lines to experiment.
The closer p is to 1 ⁄ 2, the less skewed the distribution
of the individual terms is and the smaller n has to be to get
good normal approximation. You should be able to see this for yourself,
if you experiment.
Independent and Identically Distributed Bernoulli Mixture of Normal
In this example, we use a bimodal distribution for the individual terms.
Since there are no brand name
bimodal distributions, we make one
up, the distribution of X + Y when X has
the Bernoulli distribution with success probability p and
Y has the normal distribution with mean zero and variance
σ2.
The distribution of the sum of n IID random variables having this distribution is the distribution of X + Y when X has the Binomial(n, p) distribution and Y has the Normal(0, n σ2) distribution.
Change the assignments of p, sigma,
and n in the first three lines to experiment.
The closer p is to one-half, the less skewed the distribution of
the individual terms is and the smaller n has to be to get
good normal approximation. You should be able to see this for yourself,
if you experiment.
The wiggliness caused by small sigma is less important.
Skewness
The Berry-Esseen Theorem says the rate of convergence in the central limit theorem is controlled by skewness. Every other aspect of the distribution of the individual terms has less importance. This is why it is the very skewed examples above have the worst approximation.
