What to do when you only have 7 survey responses

More than a chart would. Seven is small enough to read every word, which most surveys never get.

Research on small groups backs this up. Jakob Nielsen's rule for usability tests is that five people find most of the problems, and that "after the first study with five participants has found 85% of the usability problems" you should fix them and test again (Nielsen Norman Group). That is about watching people use a product, not about surveys. But the logic carries: the same few big problems come up again and again, so the first handful of people find most of them.

A study of open questions in interviews found something similar. When people listed everything they could think of, the first 10 interviews produced 95% of the most commonly mentioned ideas (Weller et al.). The authors' conclusion was that "if the goal is to get a few widely held ideas, a small sample size will suffice." Seven is just under that, so expect to catch the loud, common answers and miss the quieter ones.

Anything that needs a percentage. "Two of seven" is not "29% of your audience". One more person answering differently moves it by about 14 points.

The same goes for comparing groups. If three of your seven replies came from new people and four from old customers, you can't say the two groups differ. You can say what each of the seven wrote.

There's a second limit. Inbox Collective notes that "the people who respond to surveys tend to be your most opinionated readers" (Inbox Collective). Seven replies are seven people who cared enough to answer. They are real, and they are not a sample of everyone.

This takes about 20 minutes.

1. First pass: read all seven straight through. Don't write anything down.
2. Second pass: give each reply a short label for what happened to the person, such as "couldn't find the export" or "price went up at renewal". A reply can carry two labels.
3. Count the labels. With seven replies, any label that shows up twice is worth noticing. A label that shows up once is a lead, not a finding.
4. Underline the specific sentences. A reply that names a date, a tool or a moment is worth more than three that say "it was fine".

The full method, with a larger worked example, is in how to analyze open-ended survey responses.

An example

The replies and numbers below are invented to show the method. They are not from a real survey.

Say you asked seven people who tried your booking tool: "What got in the way when you set it up?"

Three of seven is not 43% of the people who try it. It is three people who all got stuck in the same place, and that's reason enough to look at that screen today. The one person who wanted a missing feature is a lead. Write it down and see if a second person says it.

Pick the two who wrote the most, or who named something specific. Send each a short, plain note: thank them, quote their own sentence back, and ask one question about it. For example: "You wrote that the second calendar was hard to find. Do you remember what you tried first?"

This is the move a bigger survey can't make. With 700 replies you'd never write to the people behind them. With seven you can, and a five-minute reply from one person often explains more than the other six combined.

Don't ask them to answer the survey again. Ask about what they already told you.

Usually, yes, but not before you've acted on these. Nielsen's advice for testing is to spread effort across several small rounds instead of one large one: "Spend this budget on 3 studies with 5 [people] each" (Nielsen Norman Group). He adds that there is little extra benefit in running more than five people through the same study (Nielsen Norman Group).

The survey version: fix or change the one thing the repeated label points at, then send the survey again to a new group and see if the same label comes back. If it does, you have a finding. If it's gone, you have a result.

If you do need a number, such as a percentage you'll show your board, that's a different job and needs a lot more replies. How many survey responses do you need works through the sizes.

You have three reasonable ways to handle seven replies.

• Read them by hand in a spreadsheet. This is the right call for seven. It costs nothing and you will remember every reply. It stops scaling at around a hundred.
• Paste them into a general AI chat. Quick for a first set of labels, but read the originals too. Can ChatGPT analyze survey responses? covers where it helps and where it merges different problems.
• Use a survey tool that reads for you. Google Forms is a common way to collect replies, and our comparison page covers how it differs on reading written answers.

Enform writes the survey from a description, you share one link, and it reads every reply and counts what people said. Every count opens to the replies behind it, so seven or seven hundred, you can check a label against the words. For seven replies you don't need it. It starts to earn its place when the next round brings in a few hundred. See pricing for what's available.