GUIDES7 MIN READ
How many responses you actually need.
Sample size is where survey planning stalls: pick a number too low and the results are noise, too high and you burn budget and goodwill chasing precision you will never use. The good news is the math is simpler than it looks, and for most surveys a handful of defaults will do. This guide covers the one formula worth knowing, the numbers to reach for, and the quieter thing that decides your sample more than any calculator: who actually replies.
01Precision, not headcount
The question “how many responses do I need” is really two questions in a trench coat: how precise must the answer be, and how confident must you be in that precision. Those are the margin of error and the confidence level, and once you fix them, the sample size falls out.
Margin of error is the ± band around your result. If 60% pick option A with a 5% margin, the true figure is somewhere between 55% and 65%. Confidence level is how often that band would contain the truth if you repeated the survey many times; 95% is the near-universal default. Tighten either one and you need more people.
02The numbers to reach for
For a large audience at the standard 95% confidence, these are the sample sizes behind each margin of error. Keep them on a sticky note and you will rarely need a calculator:
Pick the precision, read off the sample:
- ±10% margin — about 100 completed responses
- ±5% margin — about 385 completed responses
- ±3% margin — about 1,070 completed responses
- ±1% margin — about 9,600 completed responses
Notice the shape: halving the margin of error roughly quadruples the sample. Going from ±5% to ±3% is cheap; chasing ±1% is a different budget entirely. The 385 figure is the workhorse: enough to speak confidently about a whole population, small enough to be reachable.
03Why your audience size barely matters
The instinct is that a bigger population needs a bigger sample. It mostly does not. Sampling theory cares about the absolute number of responses, not the fraction of the whole, once the whole is large. The 385 that gives ±5% works for a city of 80,000 or a country of 80 million.
The exception is small, closed groups. If you are surveying a 300-person company, sampling 385 is impossible and unnecessary; you would aim for roughly 170 for the same precision, because each response now covers a bigger slice of everyone. When your population is under a few thousand, use a finite-population calculator. Above that, the round numbers in the previous section hold.
04Response rate is the real constraint
Every number so far is completed responses, not invitations sent. The gap between the two is your response rate, and it is almost always where sample plans go wrong. Decide the responses you need, then divide by the rate you honestly expect:
To net 385 responses:
- At 40% (engaged customers) — invite about 960
- At 15% (typical email list) — invite about 2,600
- At 4% (cold outreach) — invite about 9,600
- On a paid panel (~100%) — recruit about 400
DO
- Estimate response rate from your last similar send, not from hope.
- Over-recruit a little; drop-off and quality screening will trim the top.
- Track completion separately from starts so you see where people quit.
DON'T
- Don't count partials and starts as responses; report completes.
- Don't send one giant blast; stagger so you can stop at your target.
- Don't buy a list to hit a number. Junk responses widen error, not shrink it.
05When smaller is the right call
Big samples buy precise numbers. Plenty of good survey work does not need precise numbers. If you are running a usability study, sanity- checking a concept, or gathering open-text feedback, you are looking for themes and problems, and those saturate fast: twenty thoughtful responses often surface everything the next two hundred would.
The trap is subgroups. A sample of 400 feels comfortable until you slice it by four regions and two age bands and each cell holds a dozen people. If a comparison between segments is the point, size for the smallest cell you care about, not the total. That is usually what pushes a sample from hundreds into thousands.
Quick answers
How many responses is statistically significant?+
Significance is a property of a test, not a headcount, but the shorthand people want is this: for a yes/no result about a large group, around 385 completed responses gives you a 95% confidence level with a 5% margin of error. Drop to 100 and the margin widens to about 10%. There is no single magic number; the right sample depends on how precise you need to be and how small the differences you want to detect are.
Does the size of my audience change the sample I need?+
Less than most people expect. Above a few thousand people, the required sample barely moves. Sampling 385 from 20,000 and from 20,000,000 gives almost the same margin of error. Population size only matters when your group is small: for 500 people you would need roughly 220 responses for the same precision, because you are sampling a large fraction of the whole.
What is a good survey response rate?+
It varies wildly by channel. Internal or customer email surveys often land between 10% and 30%; cold external lists are frequently under 5%; paid panels approach 100% because people are compensated. Plan invitations around the rate you actually expect: to net 400 responses at a 15% rate you need to reach roughly 2,700 people.
Can a small survey still be useful?+
Absolutely. Twenty open-text responses can surface every major theme in a usability study. Fifty responses can kill a bad idea. Small samples are fine for finding problems and generating hypotheses; they are weak for precise numbers and for comparing small subgroups. Match the claim to the sample and say what the sample was.
Keep reading: Catching low-quality responses · Writing attention checks
Put it into practice.
250 responses a month free. Every control in these guides is built in.