How to write open-ended survey questions people actually answer

Because they take effort. Pew Research Center looked at 92 open-ended questions across more than 40 of its panel surveys. Closed questions mostly had 1% to 2% of people leave them blank. Open questions averaged about 18%, and individual questions ran from 3% to just over 50% (Pew).

The strongest predictor was how hard the question was to answer. Pew estimated the thinking time each one needed. Questions with a medium burden had a median blank rate twice as high as low-burden ones, and high-burden questions had three times as high. Pew also found blanks were higher on phones than on desktop.

Length of the answer asked for mattered too, but less. Multi-sentence questions had a median of 18% left blank, and one-word and short-phrase questions 13% (same Pew post). Pew did not find a reliable difference between phrase and multi-sentence questions, so don't read that as "ask for one word." Read it as "don't ask people to do a lot of work."

Pew doesn't recommend dropping open questions. It points to the texture they add. The job is to make them cheap to answer.

Four habits cover most of it.

Ask about a moment, not an opinion. Nielsen Norman Group's example is "Tell me about the last time you used the website," which gets people describing something that happened (NN/g). An opinion question makes people summarize, and they summarize into the vague middle.

Start with "what" or "how." NN/g advises favoring questions that begin with "how" or "what" over "do" or "did," and notes that leading closed questions often start with "did," "was" or "is." Those also invite a yes or a no, which is a closed question in disguise.

Ask one thing. Pew calls questions that cover more than one concept double-barreled: they "are difficult for respondents to answer and often lead to responses that are difficult to interpret," and the fix is two separate questions (Pew). SurveyCTO says the same in plainer words: asking multiple things in one question is confusing (SurveyCTO).

Cue the length. Pew's guidance is that a cue for the expected length can be explicit or implied, such as "In a sentence or two..." That tells people a paragraph isn't required.

Keep the wording neutral too. Pew has shown small wording changes can move answers a lot: in a 2003 survey, adding a clause about possible U.S. casualties moved support for military action in Iraq from 68% to 43% (Pew). A question that carries your hoped-for answer will get it back.

These pairs are made up to show the pattern. They are not from a real survey.

Notice that the rewrites are longer than a word but still one short sentence each. They name a time or a situation, which makes the answer easier to produce and easier to sort later.

Few. NN/g notes that "a survey with many open-ended questions will usually have a lower completion rate than one with more closed questions." Given Pew's blank rates, each open box also loses some people on its own.

A workable shape: one or two strong open questions about a specific moment, and a short closed question or two for the facts you need to count. Put the open question first when you can. Pew generally asks open questions early because, once closed questions have been asked, people "are much more likely to mention concepts or considerations raised in those earlier questions" (Pew).

Closed lists also miss things. In a 2008 Pew test, 58% of people picked the economy from a list as their top voting issue. Asked openly, only 35% volunteered it, and 43% of the open answers weren't on the list at all (same Pew page). If you don't know the answers yet, ask openly and build the list from what comes back.

NN/g adds one more use: when no listed option fits, or the list is too long to scan, an open text field can be the better experience for the respondent.

Read the first 30 or so answers before you decide how to group them. Give each reply a short label for what it says, and count the labels. Then reread the replies under your biggest counts, because a tidy label often hides two problems.

Here is a small worked example. The numbers are invented to show the method.

Say 40 people answer "What was the first thing you got stuck on when you signed up?" You label them like this:

The total is 40 because each answer got one label (worked out from the table). The top two rows are 25 of 40. Both are about the first five minutes, and neither is something a multiple choice list written in advance would have offered.

For the full method, see how to analyze open-ended survey responses. For the choice between open and closed in the first place, see open-ended vs closed-ended survey questions.

Three ways to get the questions written and the answers read:

• By hand in a form and a spreadsheet. You control every word. The cost is time: writing the questions, then sorting every reply yourself. See Enform vs Google Forms.
• A general AI chat. Ask ChatGPT to draft questions and to group the answers. It is fast, but check its counts against the replies.
• A survey tool built around the answers. Enform writes the survey from a description of what you're asking and who, so you can start from a drafted set of questions and then reword them using the habits above. You share one link, and it reads every reply and counts what people said. Every count opens to the replies behind it. See pricing.

If your survey is all fixed choices, any form tool will do.