Why Your Customer Feedback is Useless

Here’s the scenario: You have a minimum viable product. You’re talking to your users about it. You’re asking them questions, and they’re answering. But for some reason, it’s just not turning into usable information.

You wonder what’s going on. You imagine that perhapsyour users suck at giving good feedback or else they don’t have anything useful to say. Maybe, you think in a moment of hopeful delusion, your product is so perfect that it can’t be improved by customer feedback.

While these are all possibilities, the reality is that it’s probably not your customers’ fault. So, if you don’t seem to be getting any good data, what IS the problem? Probably one of the following things:

You’re asking the wrong questions

I’ve written before about asking customers the wrong questions, but in summary, customers are very good at giving you certain kinds of information and very bad at other kinds. For example, users are great at telling you about their problems. They can very easily tell you when something isn’t working or interesting or fun to use. What they suck at is telling you how to fix it.

Customers are great at:

• Complaining about problems
• Describing how they currently perform tasks
• Saying whether or not they like a product
• Showing you parts of a product that are particularly confusing
• Comparing one product to another similar product
• Explaining why they chose a particular method of doing something

Customers are bad at:

• Predicting their future behavior
• Predicting what other people will like
• Predicting whether they’ll pay for something
• Coming up with innovative solutions to their own or other people’s problems
• Coming up with brand new ideas for what would make a product more appealing

To take advantages of users’ strengths, have them describe things like, “Tell me about your most recent experience using the product and how that went for you.” Or ask them questions like, “What about [competitor product] do you particularly enjoy? What do you hate?” You can even ask questions like, “Of the following 5 features, which would you prefer?” You’ll want to be a lot more careful about listening to their answers to overly broad questions like, “What brand new feature would you like to see implemented?” or “What would make this product more fun to use?”

You’re asking the right questions the wrong way

It takes practice to ask good questions. Sometimes you’re too close to your product to be objective, and other times you don’t have very good moderation skills. Whatever the problem, you need to make sure that you’re being a good interviewer and not biasing the data.

One of the best ways to improve at this (aside from reading the above posts or spending years as a user researcher), is to have somebody objective and honest give you feedback on your interview skills. Get somebody else in the room who will watch you interview and tell you if you’re asking questions well. Make sure they read the above posts first though, so they’ll know what to look for.

You believe everything customers say

Why bother asking customers questions if you’re not going to listen to them? Well, you are going to listen to them, but you’re also going to verify their answers. Users lie. They don’t necessarily mean to, but they do, often because they simply don’t remember most of what typically happens when they use your product.

The easiest solution is to watch them using your product in addition to talking to them about it. Also, follow up with metrics whenever possible. Maybe they all say that they log into your website every day, but their customer history might tell a very different story. As a bonus, by watching people use your product and checking metrics, you will see your customers’ usage habits, which may vary significantly from what you expect.

You think you already know the answers

You know that saying, “when all you have is a hammer, everything looks like a nail (or a human head, depending on how sick your friends are)?” When you already have a great idea for a feature in mind, everything a customer says may lead you to believe that the feature is a good idea, even when there might be a much better solution.

It doesn’t matter what people tell you if you think you already know what you’re going to hear. When asking for feedback, you need to stay as neutral as possible, and, if you can’t, get somebody else to do the interviewing for you. Also, having more people in the room always helps, since you can compare notes about what everybody else thought the user said.

You haven’t fixed any of their old problems yet

Sometimes, you just hear the same old problems over and over and over. Do you know why that is? It’s because you haven’t fixed some very big problems! I know this sounds obvious, but I’ve worked with enough companies who completely ignored their data to know that it bears repeating. If customers keep complaining loudly about the same things, YOU SHOULD FIX THOSE THINGS. Otherwise, you’ll soon need to get some new customers.

Once you’ve fixed the big problems that are really annoying your users, you’ll be able to have a lot better discussion with them about the other issues they may be having.

You can't turn information into action items

Ok, this one’s hard. I’ve written a bit about how to improve the ROI on your user research, but there’s more to be said about this topic. The real problem is that a user test can generate hours of video and pages of notes, and it can be tricky to distill that down into a task that can be put on your scrum board and implemented by your engineers.

Here are a few suggestions to improve this:

• Hold more targeted interviews, tests, surveys, etc. For example, concentrate one series of customer development entirely on your Registration process or your Purchasing flow, and only gather data on that. By focusing, you will simply have less data to comb through, and you’ll be able to go deeper into all the problems with a particular system.
• Have several people observe interviews in real time and jot down the 5 most important things they heard during the session. Then quickly discuss everybody’s lists after the session to see if there’s consensus. This eliminates the long, costly process of going back after a series of interviews and digging through all those notes and videos.
• Use A/B testing wherever practical to answer concrete questions, like which content improves customer conversion rates or which page layout works better. This means you won’t have to spend time gathering and sifting through certain types of data, and you can focus on areas where qualitative data is more helpful. Unsurprisingly, I have already written about how to more successfully integrate your qualitative testing with A/B testing to maximize efficiency.

You’re not asking them anything!

Of course, the number one reason that people don’t get good data from their customers is that THEY’RE NOT ASKING FOR IT. All of the above techniques require that you be committed to connecting with your customers on a regular basis and getting their feedback and opinions to make your product better. If you’re not doing that, you’re ignoring a huge amount of nearly free information, and your product will likely suffer for it.

I want to hear from you!

Have you tried all of these things and still feel like you’re not getting the information you want? Is there a problem that I’ve missed that has kept you from getting good feedback? How did you learn how to do good customer development? Let me know in the comments.

Like what you read? Want to read more like it? You should really follow me on Twitter.