The product-market fit survey: reading the main benefit answers

It asks how someone would feel if they could no longer use your product. Sean Ellis built it, and the answers are "very disappointed", "somewhat disappointed" and "not disappointed."

The number people quote is 40%. Hacker Noon puts it this way: "Ellis found the magic number to be 40%," and companies that struggled to find growth almost always had under 40% of their people answer "very disappointed" (Hacker Noon). First Round Review adds that Ellis reached that figure "after benchmarking nearly a hundred startups with his customer development survey" (First Round Review).

The survey does not stop at one question. Typeform lists "the three most relevant next questions": what type of person would benefit most, what the main benefit is, and how the product could improve (Typeform). Superhuman used the same four questions (First Round Review).

A score says how far you are from 40%. It does not say what to do next. The written answers do.

Rahul Vohra, who runs Superhuman, reported that "our product-market fit score was 22%" when they started in the summer of 2017. After they narrowed to the people who loved the product, "we were at 33%." Then, "within just three quarters of our work to improve the product, the score nearly doubled to 58%" (First Round Review).

What moved it was reading the written answers. The people who loved the product most named speed, focus and keyboard shortcuts as the main benefit. The "somewhat disappointed" people who named the same benefit were the group the team "paid very close attention to, because our main benefit did resonate." Their answers pointed to what was missing, led by the lack of a mobile app (First Round Review).

That is one company's account of its own product. Treat it as a method to copy, not a result to expect.

People who have actually used the product. Typeform advises you to "filter for people who have experienced the core of the product recently," and warns that including people who signed up but never used it will unfairly lower your score (Typeform).

Learning Loop's guide lists Ellis's test for who counts: people who have used the product at least twice and in the last two weeks. It also quotes Vohra saying that around 40 respondents is where you start to get "directionally correct" results (Learning Loop). If you have fewer, this post covers what to do.

Do this in order. The first pass is the one that matters.

1. Split the replies by the first question. Three piles: very disappointed, somewhat disappointed, not disappointed.
2. Read the very disappointed pile and give each reply a short label for the benefit it names, in the person's own words where you can. "Saves me an hour on weekly reports" beats "efficiency."
3. Count the labels. The top one or two are what your fans value. That is who you are building for today.
4. Take the somewhat disappointed pile and find the ones who named that same benefit. Read their "how can we improve" answers. These people want what you offer and something is in the way.
5. Set aside the rest. Someone who names a benefit nobody else does is a different customer. Note it and move on.

Learning Loop gives one warning worth repeating: read the very disappointed people's answers in full, because AI summaries "can flatten them into generic phrases" (Learning Loop).

The numbers below are invented to show the method. They are not from a real survey.

Say 40 people answered. 14 said very disappointed, 16 somewhat disappointed and 10 not disappointed. That is 35% very disappointed, which is under 40%. Here is the main benefit question for the 14:

The shares are each count divided by 14, rounded.

Now suppose 7 of the 16 somewhat disappointed people also named the weekly report. Their improvement answers are where the work is. If four of the seven say the same thing, such as "can't pull in last month's numbers," that is a blocker worth fixing before any new feature.

• A label that hides two benefits. "Reporting" covers "saves time" and "looks good to clients." Split it when the replies underneath disagree.
• Counting only your most active people. Learning Loop warns against surveying only your most engaged people and suggests a broader set who qualify.
• Surveying at a bad moment. Typeform says to avoid sending right after "a pricing change, an outage, or a big launch."
• Treating 40% as a pass mark. It is a benchmark from one set of startups. The labels tell you more about what to do.

You have a few realistic ways to do this.

• By hand in a spreadsheet. Paste the replies in a column, add your label beside each, count. It is free and teaches you your own replies. It gets slow past a couple of hundred, and labels drift as the pile grows.
• A general AI chat such as ChatGPT. Fast at suggesting labels. Check its counts against the replies, as this post explains.
• A form tool plus your own sorting. Google Forms and Typeform collect the answers and leave the written ones to you (Enform vs Google Forms, Enform vs Typeform).

Enform writes the survey from a description, you share one link, and it reads every reply and counts what people said. Every count opens to the replies behind it, so reading the "somewhat disappointed" answers under a benefit is one click. Pricing is on the pricing page. With 40 replies, a spreadsheet is enough. Enform earns its place when you want to rerun the survey every month and compare. For a broader method on written answers, see how to analyze open-ended survey responses.

Write down the top benefit, the group it came from, and the top blocker. Fix the blocker. Send the same survey again to people who have used the product since, and compare the counts as well as the score.