Word clouds don't tell you what people said (and what to use instead)

It counts words and prints the common ones larger. Most survey tools drop filler words such as "the" and "and" first. That is all it does.

Drive Research, a market research firm, says clouds "offer a fast, low-effort way to identify the buzz words across all open-ended responses" (Gell, 2019). The same post adds that they are "less helpful in situations where the responses are multiple sentences or otherwise complicated in nature."

So a cloud fits one-word or very short answers, such as "name one word that describes us." For anything longer, it throws away most of what people wrote.

Three problems come up again and again.

They lose the sentence. Jacob Harris, then at The New York Times, wrote that "word clouds support only the crudest sorts of textual analysis, much like figuring out a protein by getting a count only of its amino acids" (Harris, 2011). "Fast" and "not fast" both feed the word "fast."

They leave the reader to guess the meaning. Harris also notes that clouds "leave them to figure out the context of the data by themselves." Robert Relihan at C+R Research puts it more bluntly: "Word clouds make the assumption that the meaning is obvious" (Relihan, 2024). Drive Research makes the same point: "There is often little context when reading a word cloud, as well."

People read size badly. A 2022 study asked 234 participants to judge word frequencies from word clouds. The authors report that "both word size scaling methods resulted in poor performance of the participants" and call clouds "a fickle measure to convey word frequencies" (Maurits, Boers and Knevel, 2022). Even the one thing a cloud promises, how often a word shows up, is hard to read off the picture.

The numbers below are invented to show the method. They are not from a real survey.

Say 40 people answered "What nearly stopped you from signing up?" The cloud would show words like "price," "time," "setup" and "confusing" in large type. Here is what the replies actually said once someone read them and gave each one a label.

"Price" is the biggest word in the cloud, and it would suggest a pricing problem. Counted by label, 11 of 40 people said they could not find the price, and 6 of 40 said it was too high. Those need different fixes: one is a clearer page, the other is a different price. The cloud merges them. It also splits "setup" and "time" into two words when 9 of 40 people described one problem.

The labels are judgment calls, so read the replies under each row before acting on a count.

Read every reply and give it a short label that says what happened, in plain words. Reuse a label when a new reply says the same thing, and add a new one when it doesn't. Put "something else" at the bottom so odd replies don't get forced into a group.

Then count each label and open the replies behind the biggest ones. A label is a summary you wrote, so rereading the replies under it is the check that keeps the count honest. Our post on how to analyze open-ended survey responses walks through this with a full worked example.

Name labels after what happened, not after a single word. "Could not find the price" is a label. "Price" is a word.

You have a few honest ways to get from replies to counts.

• A spreadsheet. Paste the replies in, add a label column, and count with a pivot table. It costs nothing and works for a few dozen replies. The cost is your time, and labels drift as you tire.
• A word cloud generator. Quick, and fine for one-word answers. For sentences it has the problems above.
• A general AI chat. Paste the replies in and ask for groups. It is fast, but it can merge two problems into one, so give each reply an ID and read the replies under any group you act on. See can ChatGPT analyze survey responses.
• Enform. It writes the survey from a description, you share one link, it reads every reply, counts what people said, and every count opens to the replies behind it. That last part is the check a cloud can't give you. Pricing is here, and here is how it compares with Google Forms, which doesn't count written answers.

If your replies are single words and you only want a picture for a slide, a cloud is fine. If you plan to change something because of the replies, count labels.