Leading survey questions: examples and how to fix them

Jotform defines it as a question that "either subtly or overtly influences a respondent to answer in a certain way because of the way the question is framed" (Jotform).

The push can come from a word in the question, from an assumption built into it, or from answer options that lean one way. The person answering usually doesn't notice. They just find one answer easier to give.

Jotform's article lists several types. Here are four of them, each with a plainer version. The leading examples are sourced from Jotform's list. The fixes are marked below.

All four leading versions are from Jotform's list. The fix in row one is Jotform's too. The fixes in rows two to four are ours, and they are one way to write it, not the only one.

Notice what the fixes share. They don't tell the person what a good answer is, and they leave room for "nothing" or "it was bad".

Yes, and by more than most people expect. Two studies show it.

Pew Research Center. Pew says "even small wording differences can substantially affect the answers people provide" (Pew). In January 2003 it asked about military action in Iraq. With the plain question, 68% favored it and 25% opposed it. When the question added U.S. casualties, 43% favored it and 48% opposed it. In a 2005 question about doctors and terminally ill patients, 51% favored "making it legal for doctors to give terminally ill patients the means to end their lives", but only 44% favored "making it legal for doctors to assist terminally ill patients in committing suicide".

Loftus and Palmer (1974). Students watched films of car accidents and were asked how fast the cars were going when they "smashed", "collided", "bumped", "hit" or "contacted" each other. The mean estimates ran from 40.8 mph for "smashed" down to 31.8 mph for "contacted" (Simply Psychology). A week later, 16 of 50 people who had been asked about "smashed" said they saw broken glass, against 7 of 50 for "hit". There was no broken glass in the film.

The second study is old, small and done on students, and Simply Psychology is a summary of it, not the paper. But the pattern matches what Pew sees in current polling: one word moves the number.

Often, yes. Pew says "less educated and less informed respondents have a greater tendency to agree with such statements", which it calls acquiescence bias. It recommends offering "a choice between alternative statements" instead.

So "Our onboarding was easy. Agree or disagree?" nudges toward agreeing. "Was onboarding easy or hard, and why?" gives both sides equal weight and asks for the reason.

Run each question through these checks.

1. Take out the adjective. Cut "excellent", "great", "new and improved". Ask about the thing, not the thing plus your opinion of it.
2. Check for an assumption. "What would bring you back?" assumes the person wants to come back. "Would anything make you consider coming back?" does not.
3. Balance the options. Use the same number of positive and negative choices, as Jotform advises. A scale of "Highly satisfied, Satisfied, Somewhat satisfied, Dissatisfied, Extremely dissatisfied" has three ways to be pleased and two ways not to be.
4. Ask one thing. Pew warns against double-barreled questions that ask about "more than one concept". "How satisfied are you with the restaurant and spa?" can't be answered honestly by someone who liked only one.
5. Add an open box. Jotform suggests an open-ended question such as "What are your thoughts on our new product features?" People who disagree with your framing can say so there.

Our post on writing open-ended questions people answer goes further on the last one. For interviews, the same problem shows up in follow-ups, covered in follow-up questions that don't lead.

The wording below is invented to show the method. It is not from a real survey.

Say you run a small app and want to know why people stopped using it. A first draft might read: "We worked hard on our new dashboard. How much did you enjoy it?" It carries your effort, an assumption that they enjoyed it, and a scale that starts at "a lot".

Rewritten: "When did you last open the app, and what were you trying to do?" It names no feature and gives no hint of the right answer. It also asks about a moment, so the reply is something you can act on.

You have three honest ways to catch leading wording before you send.

• By hand. Read the draft aloud, circle every adjective, and ask one person outside your team to answer it. It costs nothing and catches a lot. The weak spot is that you know your own product too well to hear the tilt.
• A general AI chat such as ChatGPT. Paste in the draft and ask which questions lead. It is fast and good at spotting loaded words. It doesn't know your audience, and it can be wrong about what counts as neutral for them.
• A survey tool. Google Forms, Typeform and SurveyMonkey all let you write and send questions, and the Google Forms comparison shows where Enform differs.

Enform writes the survey from a description of what you want to ask, which gives you a first draft to check with the list above. You share one link, it reads every reply, and it counts what people said. Every count opens to the replies behind it, so you can see whether the answers cluster around a word you put in the question. It doesn't promise your questions are neutral, so read the draft yourself. See pricing.