Skip to content

Guide

How to create a good survey

A good survey starts from a single clear objective and the decision its results will inform, then asks the fewest neutral, non-leading questions needed to answer it. Keep it short, order questions from general to specific, and offer balanced answer options while avoiding double-barreled or jargon-heavy wording. Before sending, pilot-test the draft with a handful of people to catch confusing questions, because every extra minute of length pushes more respondents to abandon it.

Start from one objective and the decision it drives

Before writing a single question, finish this sentence: after this survey, I will decide ___. A survey with no decision behind it collects data that nobody uses. Pick one primary objective and let it filter everything else; if a question does not help you make that decision, cut it.

Write the objective down and keep it in view while you build, so the survey does not grow into a wish list of everything you are curious about. If starting from a blank page is the hard part, an AI assistant like the one in Camaleonic Survey can turn a stated objective into a first draft of questions for you to edit down.

Write neutral, non-leading questions

A leading question suggests its own answer. 'How much did you enjoy our excellent support?' assumes you enjoyed it and praises the support in the same breath. Ask 'How would you rate our support?' instead, and let people tell you.

Keep the wording neutral and the answer options balanced: as many positive choices as negative ones, with a genuine neutral middle. And do not bundle two ideas into one question. 'Was our team fast and friendly?' is double-barreled; someone who found you fast but rude cannot answer honestly, so split it into two.

Finally, write in the plain language your respondents actually use. Internal jargon, acronyms and technical terms make people guess at what you mean, and a guessed answer is just noise in your data.

It's not a minor nuance: in a classic experiment published in Public Opinion Quarterly, Tom W. Smith found that responses about public spending were much more negative and less generous when the question used the word "welfare" than when it spoke of "aid to the poor."

Choose the right question type

Match the question type to what you need. Closed questions, such as single choice, multiple choice and rating scales, are quick to answer and easy to analyze, which makes them the backbone of most surveys. Open text questions capture nuance and surprises, but they take effort to answer and to read, so use them sparingly and place them near the end.

When you use rating scales, keep them consistent: the same direction and the same number of points throughout, so respondents are not forced to relearn the scale on every screen.

Keep it short and order general to specific

Length is the single biggest driver of drop-off. The longer a survey looks, the more people abandon it partway, and abandoned responses skew your results toward the patient few. Ask only what serves your objective, and tell people up front roughly how long it will take.

Order questions from general to specific, and from easy to sensitive. Open with a simple, engaging question to build momentum, group related questions together, and leave demographic or personal questions for the end, once some trust is established.

A study by Galesic and Bosnjak in Public Opinion Quarterly measured this in a web survey: the longer the announced duration, the fewer people started and finished the questionnaire, and questions placed later received faster answers, shorter open-ended responses, and more unanswered gaps.

Order isn't cosmetic: the Pew Research Center documents that opening questions act as context for the ones that follow and change the answers. In one of its surveys, support for legal unions between same-sex couples was 45% when the question came after the one on marriage, and 37% when asked without that prior context. They recommend opening with simple questions and leaving demographic ones for the end.

Pilot-test before you send

Send your draft to five or six people who resemble your real audience before it goes live, and ask them to think aloud as they answer. You are testing for questions that get read two different ways, options that fit no one, and the true time it takes to finish.

A short pilot catches the problems you can no longer see because you wrote the thing. Fixing a confusing question before launch costs a few minutes; discovering it after a thousand people have answered can cost you the whole dataset.

That "thinking out loud" has a name in survey methodology: it's the cognitive interview that Gordon Willis describes in Cognitive Interviewing: A Tool for Improving Questionnaire Design (Sage, 2005), and it exists precisely to see where comprehension of a question breaks down while you can still fix it.

Frequently asked questions

How long should a survey be?
There is no fixed rule, but shorter almost always performs better. Include only the questions that serve your objective; for most surveys that means staying under about five minutes, or roughly ten questions or fewer. If you genuinely need more, warn respondents up front and consider splitting it into two.
How many responses do I need for reliable results?
It depends on how precise you need to be, not on the size of your audience. For a large population, about 385 responses gives a margin of error near plus or minus 5% at the common 95% confidence level. If you only need a rough sense of direction, fewer will do; for decisions with real consequences, aim higher.
What is a leading question, and why should I avoid it?
A leading question nudges people toward a particular answer through its wording, for example by praising the thing it asks about or assuming a feeling. It biases your results toward what you hoped to hear rather than what people actually think. Keep the wording neutral and the answer options balanced so the data reflects real opinions.

Sources

  1. Smith, T. W. (1987). That Which We Call Welfare by Any Other Name Would Smell Sweeter. Public Opinion Quarterly, 51(1), 75-83The classic experiment on how much an answer shifts with the word you choose.
  2. Galesic, M. & Bosnjak, M. (2009). Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey. Public Opinion Quarterly, 73(2), 349-360The measurement of what length does to participation and to answer quality.
  3. Willis, G. B. (2005). Cognitive Interviewing: A Tool for Improving Questionnaire Design. SageThe method behind asking someone to think out loud as they answer.
  4. Pew Research Center, Writing Survey QuestionsReference methodology on neutral wording, question order and balanced answer options.

Build your survey with Camaleonic Survey

AI to draft it, your brand on every screen, and live results as responses arrive.

Start free