Feature request survey replies: sort by the situation, not the ask

Because the person is proposing a fix, and you asked for a fix. They know their own day better than they know your product.

Canny, a feedback-board vendor, puts it this way: "What a lot of people do when they give feedback is suggest solutions." It adds that "You’re not going to know whether this “solution” really solves a problem until you identify the problem itself" (Canny).

Productboard, a product-management vendor, makes the same point: "People don’t talk about problems naturally. Their thinking is framed by the existing solution" (Productboard). Both are companies that sell feedback tooling, so read them as practitioner opinion, not research findings. The idea itself is common across product writing.

The example people reach for is dark mode. Perspective AI, which sells an interview tool, notes that someone asking for a toggle "might actually be working night shifts and fighting eye strain" and that "the toggle is their guess, not their need" (Perspective AI).

Two different problems end up under one label, and the label then reads like a plan.

Say five people write "add a Slack integration". You label them "Slack integration" and count five. Now the line "Slack integration: 5" sits on a slide. Nobody reopens the replies. The title has become the requirement.

The replies may say something else. One person wants a ping when a reply lands. One wants to forward answers to a teammate. One is tired of checking a dashboard. A Slack integration serves the first, partly serves the second, and misses the third.

Name each group after what the person was doing or stuck on, in a few words. Not the feature they named.

• Bad labels: "Slack integration", "export", "dark mode".
• Good labels: "needs to know when a reply lands", "shares answers with someone outside the tool", "reads results late at night".

Three rules keep the labels honest:

1. One label per reply, plus a second if the person asked for two different things.
2. Describe the situation in the person's words where you can. "Sends a weekly summary to a boss" beats "reporting".
3. Keep an "unclear" label. "Just make it better" is unclear. Count it as unclear.

The numbers below are invented to show the method. They are not from a real survey.

Say 30 people answered "What is the one thing you wish this did?" and 12 of them asked for an export. Read those 12 and sort by what each one says about why:

The shares are each count divided by 12, rounded.

"Export: 12" would have started an export project. The sorted list shows the biggest group wants to hand results to a person without a login, which a shareable page or a short deck might solve better than a file. Only three of the 12 want the spreadsheet.

Ask for a moment, not a wish. Two lines from the sources are worth copying into the survey.

• "Tell me about the last time you needed this." Perspective AI says "Specific recent stories beat hypotheticals" (Perspective AI).
• Ask what they would do if the feature existed, and what they do today without it (eChai Ventures).

A follow-up box under the "one thing you wish this did" question costs the reader ten seconds and gives you the situation in their own words. Without it you are left guessing from a one-line ask.

The largest two, plus any label that surprises you. Open the replies under each and read them with the label in mind.

You are checking two things. Do all the replies say the same thing? And would one sentence from a reply, read out loud, make the point better than the number does?

If a label hides two situations, split it. Do this before the counts go into a roadmap doc, because after that nobody rereads.

You have a few realistic ways to do this.

• By hand in a spreadsheet. Paste replies in one column, put your label next to each, count the labels. Free, and the best way to learn your own replies. It gets slow past a couple of hundred, and labels drift as the pile grows.
• A general AI chat such as ChatGPT. Quick for suggesting labels. Check its counts against the replies before you trust them, as this post explains.
• A form tool plus your own labelling. Google Forms and Typeform collect the answers, and text answers still need sorting (Enform vs Google Forms, Enform vs Typeform).

Enform writes the survey from a description, you share one link, and it reads every reply and counts what people said. Every count opens to the replies behind it, so rereading under a label is one click. Pricing is on the pricing page. If you have 30 replies, a spreadsheet is enough. Enform earns its place when the pile is too big to relabel by hand. If you already have a list of ideas and need people to rank them, the feature prioritization survey is the better fit.

Match each situation to its own fix, and keep the replies next to it.

• A group of people who can't share results points to sharing, not exporting.
• A group who need a ping points to notifications.
• An "unclear" pile that keeps growing points to the survey, not the product. Ask a better question next time.

Then run the same survey again after you ship and compare the labels. For the same trap on cancellation replies, see how to read churn survey replies.