How to test a survey before you send it

Most survey mistakes are easy to find before launch and hard to undo after it. A confusing question, a route that skips half the survey, a missing answer option: once real responses arrive, the mistake is part of your data, and the only remedy is another round.

This checklist is the testing that catches them, in roughly the order worth doing it. Most steps take minutes. The pilot takes longer and is the easiest to skip, and it is where the surprises usually are.

1. Read it through as a respondent

Open the full preview and take the survey the way a respondent will, from the welcome screen to the closing one, without editing as you go. Note anything you had to read twice. Then check that:

• The welcome screen says what the survey is for and roughly how long it takes.
• Every question asks one thing, in words your audience uses. The guide to writing unbiased questions covers what to look for.
• Answer options cover everyone and do not overlap.
• Only the questions you truly need are required.

In Plumeform, the builder's pre-flight panel works alongside you. It stops a publish for anything that would break the survey, flags problems worth fixing, points at wording worth a second look, and shows the estimated time and reading grade.

2. Walk every route

If the survey branches, list every route and walk each one in the preview: every answer that changes the path, a respondent who matches no rule, and someone who goes back and changes an earlier answer. Check the endings as well: the normal completion, any early end, and, for studies with eligibility criteria, the screen-out message. The article on planning skip logic shows how to turn a flow into a test table.

3. Try it on a phone

Open the survey on an actual phone, not a narrow desktop window. Long option lists, rating grids, sliders and images are where small screens cause trouble. Type into the text fields with the on-screen keyboard open, and check that nothing you need sits underneath it.

If the survey will live inside your website, test it there too, in the page and the container it will actually sit in. An embed that looks fine on its own link can be squeezed or clipped by the layout around it; the help article on embedding a survey lists what to check.

4. Pilot it with a few real people

Ask a few people from your audience, or as close to it as you can get, to take the survey while you watch or while they think aloud. You are not collecting data. You are looking for the question somebody reads differently from how you meant it, and one usually turns up.

Afterwards, ask them three things. Was anything confusing? Was there a question you could not answer honestly with the options given? Did anything feel too personal or out of place? Their answers point straight at what to fix.

Time them. Your own run-through is faster than anyone else's, because you wrote the questions. If their times are well over your budget, cut now; the article on survey length explains how to decide what goes.

For colleagues without an account, a Plumeform reviewer link opens a read-only preview of the draft. Treat the link as access, and turn it off when the review is over.

5. Do a live test run

Once the survey is published, go through the live link in test mode by adding ?test=1 to it, or &test=1 if the link already has a query string. Test mode works on every plan. The response is stored but flagged as a dry run, kept out of your results and exports, and not counted against your monthly responses. The results page shows how many test responses there are, with a button to delete them.

A test run proves the published survey loads and submits. It does not prove that everything around the survey behaves the same way with real traffic, such as quotas, duplicate checks and deliveries to other tools, so check those on their own terms.

6. Check the data, not just the screens

Screens can look right while the data comes out wrong: a question whose answers you cannot chart, an open question where you needed a number, a source you meant to track that never arrives. Before launch, write down how you will analyze each question and make sure the survey will give you that.

The surest check is one real response. Submit one yourself on the live link without test mode, open it in your results, and download the CSV. It counts as one of your monthly responses. Once you have checked it, exclude it in the Responses view with a reason such as “My own test”. An excluded response leaves your results and exports and is never deleted.

7. Decide your quality rules now

If your results will depend on removing careless or fraudulent responses, write the rules down before any data arrives: what counts as too fast, what happens after a failed attention check, how you will treat identical answers down a grid. Rules chosen after seeing the results tend to favor the results you hoped for.

Plumeform flags unusually fast responses and repeated answer patterns on every plan; attention checks are part of the Pro research tools. The guides to attention checks and low-quality responses cover how to write the checks and what to do with the flags.

8. Check the settings around the survey

A good questionnaire can still fail on a setting. Before sharing the link, check:

• Open and close dates, and any cap on the number of responses.
• Duplicate protection. One response per browser is a convenience check that clearing browser storage defeats; for panel studies, the check on the panel participant ID is the stronger one.
• The closing screen, and any redirect or completion code a panel needs.
• What happens after the closing screen. On Free, a finished survey moves on to plumeform.com after a few seconds, with a link to stay; embedded surveys, panel surveys that show a completion code and surveys that redirect somewhere of their own are left alone.
• Email alerts for new responses, if you want them, going to an address someone reads.
• Your monthly allowance. On Free that is 100 responses a month; if you reach it, collection continues for the rest of that day, up to 10% over, and then pauses until the next month or an upgrade.

9. Launch in stages and watch the first responses

If you can, send to a small first batch, wait for a few dozen responses, and look before sending the rest. In the results summary, compare views with starts and completed responses, check the median time, and read the list of questions where people leave. A question many people abandon, or a median time far above your estimate, is cheap to fix now and expensive later.

You can still fix things once collection has started: in Plumeform, publishing saves a new version, and every response stays linked to the version its respondent answered. Keep changes to fixes, though. Reword a question halfway through and the answers before and after are answers to two different questions.