Inference
Confidence Intervals
In Central Limit Theorem we saw that every random sample gives a slightly different mean, and that the standard error measures how much these means vary. This post shows how to turn one sample into an honest statement about the population: not just a single number, but a range of plausible values.
Point estimates and confidence intervals
A point estimate uses a single value to estimate a population parameter, such as the average session time on a website or the percentage of customers who would recommend it. It is easy to report, but it says nothing about how far off it might be.
A confidence interval gives a range of values that likely contains the population parameter. It expresses the uncertainty that comes from random sampling. Suppose a random sample of 100 sessions on a website has an average session time of 4.0 minutes. The point estimate is 4.0 minutes. The 95% confidence interval is 3.51 to 4.49 minutes.
A confidence interval for a mean estimates the average value of a group, such as the average session time on a website. A confidence interval for a proportion estimates a percentage or fraction, such as the share of visitors who buy something.
Components of a confidence interval
- Sample statistic. The estimate from your sample data, for example the sample mean session time.
- Margin of error. The maximum expected difference between the sample estimate and the true population parameter. You calculate it from the standard error and a z-score.
- Confidence level. How confident you want to be that the method captures the true parameter. It is commonly set at 90%, 95% or 99%.
confidence interval = sample statistic ± z × standard errorz × standard error is the margin of error.
The z-score depends only on the confidence level you choose. A higher confidence level needs a larger z-score, so the interval gets wider.
| Confidence level | z-score |
|---|---|
| 90% | 1.645 |
| 95% | 1.96 |
| 99% | 2.576 |
Construct the confidence interval
You can build a confidence interval in four steps. The example uses the session-time data from above.
- Identify a sample statistic. Calculate the estimate from your sample. The mean session time of 100 sessions is 4.0 minutes, with a sample standard deviation of 2.5 minutes.
- Choose a confidence level. Decide how confident you want to be, commonly 90%, 95% or 99%. Here we choose 95%, which gives a z-score of 1.96.
- Find the margin of error. Multiply the z-score by the standard error. The standard error is 2.5 / √100 = 0.25 minutes, so the margin of error is 1.96 × 0.25 = 0.49 minutes.
- Calculate the interval. Add the margin of error to the sample statistic and subtract it from the sample statistic: 4.0 ± 0.49, or 3.51 to 4.49 minutes.
A larger sample would shrink the standard error and make the interval narrower, while asking for 99% confidence would make it wider.
z-score or t-score?
The steps above use the z-score. When you estimate a mean and only know the sample standard deviation, which is the usual case, the more accurate choice is the t-score, especially for small samples (roughly n < 30). It comes from the t-distribution, which has heavier tails and gives a slightly wider interval. For large samples such as n = 100 the two are almost identical (1.984 versus 1.96), so z is a good approximation. For a proportion, use z.
The same four steps work for a proportion. Only the standard error changes: it is calculated from the sample proportion p̂ and the sample size n.
standard error = √( p̂ × (1 − p̂)n )p̂ is the sample proportion and n is the sample size.
Suppose 40 of 500 sessions end in a purchase, so p̂ = 0.08, a conversion rate of 8%. The standard error is √(0.08 × 0.92 / 500) ≈ 0.012. At the 95% level the margin of error is 1.96 × 0.012 ≈ 0.024, so the interval runs from 5.6% to 10.4%.
Interpreting a confidence interval
A 95% confidence interval means that if you take many random samples and build intervals the same way, about 95% of those intervals will contain the true population mean. The confidence level describes the long-term success rate of the method, not a single interval.
Common misinterpretations
- “There is a 95% probability that the population mean is in this interval.” Incorrect. The population mean is a fixed number, not a random variable. Once an interval is calculated, it either contains the mean or it does not.
- “95% of the population values fall in the interval.” Incorrect. The interval relates only to the estimate of the mean, not to individual sessions.
- “95% of all possible sample means fall in the interval.” Incorrect. Sample means vary, and many of them fall outside any one interval.
Prediction interval
A confidence interval estimates where the population mean lies. A prediction interval answers a different question: where might a future individual observation fall? It is always wider, because it must cover the spread between individual values as well as the uncertainty in the mean.
How to explain it to stakeholders
“We looked at a random sample of 100 sessions. Their average time is 4.0 minutes, but that is only an estimate, because a different sample would give a slightly different number.”
“So we report a range: we are 95% confident that the true average session time lies between 3.51 and 4.49 minutes. This means that if we repeated the study many times, about 95 out of 100 such ranges would contain the true average.”
“If this range is too wide for the decision at hand, for example when setting a target for session time, the answer is a larger sample, not a different calculation.”
Key takeaway
Confidence intervals quantify the uncertainty that comes from sampling and are a fundamental tool of statistical inference. A single estimate can mislead, a range shows how far it can be trusted, and stating its meaning correctly keeps stakeholders from making decisions based on a misunderstanding.