Sampling

Sampling Methods

A population is every member of the group you want to understand. A sample is the part of it you actually collect data from. A sample is only useful if it represents the population, and how you pick it decides that. Sampling methods fall into two families: probability sampling, which uses random selection, and non-probability sampling, which does not.

The sampling process

Sampling is a sequence of decisions. Each step builds on the one before it.

• Identify the target population. Define the complete set of elements you want to study, such as customers who purchased a specific product.
• Select the sampling frame. Create a specific list of individuals or items from the target population to draw your sample from.
• Choose the sampling method. Decide between probability sampling (random selection) for representativeness and non-probability sampling (convenience or preference-based).
• Determine the sample size. Balance precision and resource constraints to decide how many individuals or items to include.
• Collect the sample data. Gather data from the selected sample, for example by conducting surveys.
• Use the data to make informed decisions. Analyze the sample data to infer insights about the population and guide business or research decisions.

Carefully executing each step helps ensure that your sample reflects the population, which makes your conclusions more reliable. The next sections look at step 3 in detail.

Probability sampling

In probability sampling, selection is random. Every member of the population has a known chance of being picked, which keeps bias low and makes the sample more likely to represent the population. There are four common methods.

Simple random sampling

Every member of the population has an equal chance of being selected, often using a random number generator. Example: drawing 200 customers at random from a full customer list.

Stratified random sampling

The population is divided into groups called strata (for example by age or gender), and a random sample is taken from each group. Example: sampling customers separately in each country, so that small countries are not missed.

Cluster random sampling

The population is divided into clusters, some clusters are picked at random, and all members of the picked clusters are included. Example: choosing 5 of 40 stores at random and surveying every customer in those stores.

Systematic random sampling

Members are put in order, a random starting point is chosen, and then every n-th member is selected. Example: starting at a random invoice and taking every 10th one.

Non-probability sampling

Non-probability sampling does not use random selection, so not every member of the population has an equal chance of being picked. That means a high risk of sampling bias. The methods are cheaper and more convenient, which makes them useful when budget or time is limited, or for exploratory studies where the goal is a first understanding of a population rather than conclusions about all of it.

• Convenience sampling. Easily accessible members are selected, such as shoppers who happen to be in your store. It often causes undercoverage bias, because less accessible groups are missed.
• Voluntary response sampling. People choose to take part, such as an online poll. It often causes nonresponse bias, because people with strong opinions are more likely to respond.
• Snowball sampling. A few participants recruit others. It helps reach hard-to-reach populations, but the sample can end up with similar characteristics, which biases it.
• Purposive sampling. Participants are selected for specific needs of the study, such as interviewing only power users. It focuses on one group but excludes others, so the results are hard to generalize.

Common sampling mistakes

Choosing a sampling method is only one part of good sampling. Mistakes can happen when defining the population, selecting participants, or interpreting the results. A large sample does not automatically produce reliable conclusions: systematic bias can remain even when thousands of observations are collected.

Sampling from the wrong population

• The mistake: The sample does not match the population the analysis is supposed to describe.
• Example: A retailer surveys customers who shop in its online store and uses the results to describe all its customers, including those who shop exclusively in physical stores.
• How to avoid it: Define the target population before collecting data. Make sure the sampling frame covers the people or entities you want to understand.

Choosing convenience over representativeness

• The mistake: Selecting observations simply because they are easy to access.
• Example: A company surveys customers who happen to visit one store on a weekday afternoon. Customers who shop at other locations or on weekends may have different preferences.
• How to avoid it: Use a probability sampling method when you need to make statistical inferences about the wider population. If convenience sampling is necessary, clearly state its limitations.

Assuming a large sample is always better

• The mistake: Believing that increasing the sample size automatically eliminates bias.
• Example: A survey receives 10,000 responses through a voluntary online poll. If people with strong opinions are more likely to participate, the results can still be biased.
• How to avoid it: Evaluate how the sample was selected, not just how many observations it contains. A larger sample can reduce random sampling error, but it does not automatically correct systematic bias.

Ignoring nonresponse

• The mistake: Treating the people who respond as representative of everyone who was selected.
• Example: A customer satisfaction survey receives responses mainly from highly satisfied customers, while dissatisfied customers rarely participate.
• How to avoid it: Follow up with nonrespondents where possible and compare respondents with nonrespondents using available characteristics. Consider weighting adjustments when suitable information is available.

Forgetting important subgroups

• The mistake: Collecting too few observations from groups that matter to the analysis.
• Example: A retailer wants to compare purchasing behaviour across five countries, but most survey responses come from Germany. Estimates for smaller markets may be unreliable.
• How to avoid it: Consider stratified sampling to ensure relevant groups are included. If the sample intentionally oversamples smaller groups, use appropriate weights when estimating population-level results.

Treating clustered observations as independent

• The mistake: Assuming that observations from the same group are statistically independent.
• Example: A researcher measures wool yield from 500 sheep in one flock. The sheep share weather conditions, pasture, and management, so their outcomes may be correlated.
• How to avoid it: Identify the unit of observation and how the data is grouped. Depending on the study design, consider sampling across multiple flocks or using a statistical model that accounts for clustering.

Generalizing beyond the sample

• The mistake: Drawing conclusions about a wider population than the data can support.
• Example: A survey of customers from one luxury fashion brand is used to make claims about all fashion consumers.
• How to avoid it: State clearly which population the results describe. Explain limitations in coverage, selection, and uncertainty before making recommendations.

A practical checklist

Before interpreting sample data, ask:

• Does the sample match the target population?
• Who could not be selected, or was unlikely to participate?
• Is the sample size sufficient for the analysis and relevant subgroups?
• Are observations independent, or are they grouped?
• Do the conclusions stay within the limits of the data?

How to explain it to stakeholders