Almost every company I talk to wants to test their products, get customer feedback, and iterate based on real user metrics, but all too often they have some excuse for why they just never get around to it. Despite people's best intentions, products constantly get released with little to no customer feedback until it's too late.
I'm not trying to promote any specific methodology for testing your products or getting customer feedback. Whether you're doing formal usability testing, contextual inquiries, surveys, a/b testing, or just calling up users to chat, you should be staying in contact with customers and potential customers throughout the entire design and development process.
To help get you to stop avoiding it, I've explored six of the most common stupid excuses for not testing your designs and getting feedback early.
Excuse 1: It's a design standard
You can't test every little change you make, right? Can't you sometimes just rely on good design practices and standards? Maybe you moved a button or changed some text. But the problem is, sometimes design standards can get in the way of accomplishing your business goals.For example, a few months ago at a talk given by Bill Scott, he talked about a developer who had a/b tested the text on a link. One option read, "I'm now on Twitter." The second read, "Follow me on Twitter." The third read, "Click here to follow me on Twitter." Now, anybody familiar with "good design practices" will tell you that you should never, ever use the words "click here" to get somebody to click here. It's SO Web 1.0. But guess which link converted best in the a/b test? That's right. "Click here" generated significantly more Twitter followers than the other two. If that was the business goal, the bad design principle won hands down.
Does this mean that you have to do a full scale usability test every time you change link text? Of course not. Does it mean you have to use the dreaded words "click here" in all your links? Nope. What it does mean is that you should have some way to keep an eye on the metrics you care about for your site, and you should be testing how your design changes affect customer behavior, even when your changes adhere to all the best practices of good design. So, to put it simply: prioritize what you care about and then make sure you test your top priorities.
Excuse 2: Company X does it this way
I can't tell you how many times I've heard, "Oh, we know that will work. Google/Facebook/Apple does it that way." This is the worst kind of cargo cult mentality. While it's true that Google, Facebook, and Apple are all very successful companies, you aren't solving exactly the same problem that those companies are, you don't have exactly the same customers that they do, and you don’t know if they have tested their designs or even care about design in that particular area. You are, hopefully, building an entirely different product, even if it may have some of the same features or a similar set of users.Is it ok to get design ideas from successful companies? Of course it is. But you still need to make sure your solutions work for your customers.
I previously worked with a company that had a social networking product. Before I joined them, the company decided that, since other companies had had good luck with showing friend updates, they would implement a similar feature, alerting users when their friends updated their profiles or bought products. Unfortunately, the company's users weren't very interested in the updates feature as it was implemented. When we finally asked them why they weren't using the feature, the users told us that they would have been very interested in receiving an entirely different type of update. Of course, if the company had connected with users earlier in the process, they would have rolled the feature out with the right information and gotten a much more positive reaction on launch.
Another thing to remember is that just because a company is successful and has a particular feature doesn't mean it's that exact feature that makes them successful. Google has admitted that the "I'm Feeling Lucky" button loses them page views, but they keep it because they, and their customers, like the feature. That doesn't mean it's a good business plan for your budding search engine startup to adopt a strategy of only providing people with the equivalent of the "I'm Feeling Lucky" button. In fact, this is a great example of why you might need to employ multiple testing methods: qualitative (usability, contextual inquiry, surveys), to find out if users find the feature compelling and usable, and quantitative (a/b, analytics), to make sure that the feature doesn't bankrupt you.
The bottom line is, it doesn't matter if something works for another company. If it’s a core interaction that might impact your business or customer behavior, you need to test new features and designs with your customers to make sure that they work for you. Obviously, you also need to make sure that you’re not violating anybody’s IP, but that’s another blog post.
Excuse 3: We don't have time or money
The fact is you don't have time NOT to test. As your development cycle gets farther along, major changes get more and more expensive to implement. If you're in an agile development environment, you can make updates based on user feedback quickly after a release, but in a more traditional environment, it can be a long time before you can correct a major mistake, and that spells slippage, higher costs, and angry development teams.I know you have a deadline. I know it's probably slipped already. It's still a bad excuse for not getting customer feedback during the development process. You're just costing yourself time later. Besides, it’s not as expensive as you think! You don’t always have to do some enormous user test in a formal lab with one way glass. We advocate cheap and dirty Fast Insight Testing, which you can learn about here.
Excuse 4: We're new; We'll fix it later
I hear this a lot from startups, especially agile ones, that are rushing to get something shipped, and it's related to the previous excuse. Believe me, I do understand the pressures of startups. I know that, if you don't ship SOMETHING you could be out of business in a few months. Besides, look at how terrible some really popular sites looked when they first started! You have to cut something, right?Great. Cut something else. Cut features or visual polish. Trust me, people will forgive ugly faster than they'll forgive completely unusable or confusing. Whatever you decide to cut, don't cut getting customer feedback during your development process. As I mentioned in Excuse 3, you can do it faster than you think. But more importantly, if you ship something that customers can't use, you can go out of business almost as fast as if you hadn't shipped anything.
Potential users have a lot of options for products these days. If they don't understand very quickly all the wonderful things your product can do for them, they're going to move on. Take a few hours to show your ideas to users informally, and you will save your future self many hours of rework.
Excuse 5: It's my vision; users will just screw it up
This can also be called the "But Steve Jobs doesn't listen to users..." excuse. Except, that's not true. Recently, in an interview with the New York Times, when asked why the Nano got a camera while the Touch didn't, Jobs responded, "Originally, we weren’t exactly sure how to market the Touch...What happened was, what customers told us was, they started to see it as a game machine. We started to market it that way, and it just took off. And now what we really see is it’s the lowest-cost way to the App Store, and that’s the big draw. So what we were focused on is just reducing the price to $199. We don’t need to add new stuff. We need to get the price down where everyone can afford it." Apparently, this myth that Jobs doesn't listen to customer feedback or make decisions about product features based on talking to users is overblown.The fact is, understanding what your users like and don't like about your product doesn't mean giving up on your vision. You don't have to make every single change suggested by your users . You don't have to sacrifice a coherent design to the whims of a focus group.
What you should do is connect with your users or potential users in various different ways - user tests, contextual inquiry, metrics gathering, etc. - to understand whether your product is solving the problem you think it is for the people you think are your customers. And, if it's not, it's a good idea to try to understand why that is and develop some ideas for how to fix it.
Besides, even if Steve Jobs never listened to a single customer in his entire life, how many wanna-be Steves do you think there out there whose companies failed miserably because nobody wanted to use their products?
Excuse 6: It's just a prototype to get funding
This is an interesting one, since I think it's a fundamental misunderstanding of the entire concept of customer research. When you're building a prototype or proof of concept, you still need to talk to your customers. The thing is, you may have an entirely different set of customers than you thought you did.Maybe you think the target market for your new networked, Wi-Fi lunchbox is 11-13 year old girls, but they're not going to pay you to build the first million units and get them into stores. Your first customers are the venture capitalists or the decision makers at your company or whoever is going to look at your product and decide whether or not to give you money.
Even if they're not your eventual target market, it's probably a good idea to spend some time talking with whomever you're trying to get to fork over the cash. I'm not saying you should change your entire product concept based on this feedback. I mean, if you really want to start the company on your credit cards and a loan from your mom, don't change a thing! The important take away here is that you may have different audiences at different points in your company's life. And the best way to find what they all want is to talk to them!
