Survey terms, explained.

35 terms with definitions, examples and common mistakes. Use them while planning a study, reading a report, or checking what a result means.

Sampling, precision and design

How many people, how precisely they describe anybody else, and what makes a comparison between two groups mean something.

Sample sizen: The number of responses in your analysis. It affects the precision of your estimates.
Margin of errorsampling error, MoE: An estimate of sampling uncertainty around a survey percentage, stated at a chosen confidence level.
Confidence intervalconfidence level, CI: The range your result is consistent with, and the confidence level is how often that kind of range would contain the truth.
Statistical significancep-value, significant difference: A statement about how often chance alone would produce a difference at least as large as the one you found, if there were really no difference.
Cross-tabulationcrosstab, contingency table: A table that breaks down answers to one question by answers to another.
Weighting: Adjusting how much each response counts, so the sample's composition matches known population figures.
Random assignmentrandomisation, A/B test, randomized assignment: Allocating participants to conditions by chance, which is what lets a difference between groups be read as an effect.

Who answers, and who does not

How recruitment and participation affect who is represented in your results.

Response rate: The share of invited people who completed the survey.
Nonresponse bias: The error that arrives when the people who did not answer differ from the people who did, in a way that matters to your question.
Selection bias: Error introduced by how people came to be in your sample, when that process is related to what you are measuring.
Sampling frame: The actual list your sample is drawn from, which is rarely the same thing as the population you want to describe.
Screenerscreening questions, eligibility questions: The questions at the start that decide whether somebody belongs in the study, and route them accordingly.
Quotaquota sampling, quota cell: A cap on how many responses you will accept for a segment, so one group cannot fill the whole sample.
Completion ratedrop-off rate, partial response: The share of people who started your survey and reached the end; its inverse is drop-off.

Writing the questions

Question formats, wording problems and the effects they can have on answers.

Likert scalerating scale, agreement scale: A set of ordered answer options, usually four to seven, running between two poles or up from none of something.
Ordinal data: Answers with a known order but unknown spacing, which is what almost every rating question produces.
Matrix questiongrid question, battery: Several statements sharing one set of answer columns, shown as a grid.
Double-barrelled question: One question asking about two things, so a single answer has to cover both.
Leading question: A question whose wording points at one answer, so the result partly measures the wording.
Social desirability bias: The tendency to answer the way that reflects well on you rather than the way that is true.
Net Promoter ScoreNPS: The share of people scoring 9 or 10 on a recommend question minus the share scoring 0 to 6, in percentage points.

Research methods

Methods that are a set of questions with their own analysis: which features matter, what a price feels like, and what people prefer.

Kano modelKano analysis, Kano survey, Kano questionnaire: A way to sort product features by how people feel when a feature is there and when it is not: must-have, performance, attractive or indifferent.
Price sensitivity meterVan Westendorp, Van Westendorp price sensitivity meter, PSM: Four price questions about one offer (too cheap to trust, a bargain, getting expensive, too expensive) whose answer curves cross at an acceptable price range.
Paired comparisonpairwise comparison, head-to-head comparison, Bradley-Terry: Asking which of two options someone prefers, over many pairs, and turning the wins into a score for every option.
Q methodologyQ-sort, Q sort, forced distribution: Asking people to sort a set of statements onto a fixed grid, from most agree to most disagree, so that every placement is a choice against the others.
Card sortingcard sort, open card sort, closed card sort, hybrid card sort: Asking people to put a set of cards (pages, products, topics) into groups, to learn how they expect things to be organised.
First-click testingfirst click test, click test, click map, heat map: Showing people a design and asking where they would click first to do something, then timing and mapping that click.
Five-second test5-second test, first impression test: Showing a design or a message for a few seconds, then asking what people remember or think it is for.
Conjoint analysischoice-based conjoint, CBC, discrete choice, trade-off analysis: Asking people to choose between whole offers that differ on several attributes at once, then working out how much each attribute and level drove the choice.
Implicit Association TestIAT, D score, implicit attitudes: A timed sorting task that measures how much faster people pair two concepts with one attribute than the other, scored as a D score.

Quality and ethics

Reviewing response quality and explaining participation and privacy.

Attention checkinstructional manipulation check, IMC: An item with a knowable correct answer, used as one signal that somebody is reading rather than clicking.
Straight-liningnon-differentiation: Selecting the same answer across every row of a question battery.
Informed consent: Agreement to take part given before anything is collected, by somebody who has been told what taking part involves.
De-identificationanonymisation, pseudonymisation: Removing or replacing the details that could identify a respondent, so what you hold cannot be traced back to a person.
Anonymity thresholdminimum group size, minimum reporting size, small cell suppression: The smallest number of people a result may describe. Any summary of fewer is hidden, so nobody can be picked out of it.

Calculate sample size, margin of error, significance and NPS, or generate scale labels, with our free survey tools. The long-form reasoning lives in the research guides.