The law of large numbers and the central limit theorem

 Let’s talk about the law of large numbers and the central limit theorem in a way that is easy to understand. As we know that the law of large numbers is a statistical concept that states that as you take more and more samples from a population, the average value of those samples will tend to get closer and closer to the true average value of the population. To put it simply, the more data you collect, the more accurate your estimate of the true population value becomes.

For example, let's say you wanted to estimate the average height of all the people in a city or within a country. You could take a sample of 10 people and calculate their average height. Then, you could take another sample of 10, 100, or more people and calculate their average height. If you keep doing this, taking more and more samples and calculating their average height, you'll start to notice that the average of all your sample averages will tend to get closer and closer to the true average height of the entire population of your target population. There is another example to help us illustrate these concepts further. Let's say you wanted to estimate the average number of hours of sleep that college students get per night. You could take a small sample of 10 students and calculate their average number of hours of sleep. However, this small sample may not be representative of the entire college student population, and you may not get an accurate estimate of the true average. If you were to take a larger sample of 100 students, you'd be more likely to get a better estimate. And if you were to take an even larger sample of 1000 students, your estimate would likely be even more accurate. This is because as you take larger samples, you are more likely to capture the diversity of the population, and your estimate becomes more reliable.

Now, let's move on to the central limit theorem. This theorem states that, regardless of the shape of the original population distribution, the distribution of sample means will tend to be normally distributed as the sample size increases. In other words, if you take a large enough sample size from any population, the distribution of sample means will start to look like a bell curve, with most of the data clustering around the mean value.

To understand this better, let's go back to our example of estimating the average height of all the people in a city. Let's say that the population distribution of heights is not normally distributed, but instead is skewed to the right (there are more people who are taller than average). If you take a small sample size, your sample average might be skewed as well. However, as you take larger and larger sample sizes, the distribution of sample means will start to look more and more like a normal distribution, with most of the sample means clustering around the true population average. Another example for the central limit theorem, let's say you wanted to estimate the average weight of peanuts in a box. If you were to weigh every peanuts in the box, you'd get a good estimate of the true average weight. However, this would be time-consuming and impractical. Instead, you could take a sample of 20 peanuts and calculate their average weight. If you were to repeat this process many times, taking different samples of 100 peanuts each time, you'd find that the distribution of sample means starts to look like a bell curve. Even if the weights of individual peanuts were not normally distributed, the distribution of sample means would tend to be normally distributed as the sample size increases, as per the central limit theorem.

Therefore, to summarize, the law of large numbers tells us that the more data we collect, the more accurate our estimates of population values become, while the central limit theorem tells us that regardless of the shape of the original population distribution, the distribution of sample means will tend to be normally distributed as the sample size increases. I hope these explanations and examples help to clarify the concepts of the law of large numbers and the central limit theorem.

Reference

Yakir, B. (2011). Introduction to statistical thinking (with R, without Calculus). The Hebrew University of Jerusalem, Department of Statistics