Tiny Tests: User Research You Can Do NOW!

There’s a lot of advice about how to do great user research. I have some pretty strong opinions about it myself.

But, as with exercise, the best kind of research is the kind that you actually DO.

So, in the interests of getting some good feedback from your users right now, I have some suggestions for Tiny Tests. These are types of research that you could do right this second with very little preparation on your part.

What Is a Tiny Test?

Tiny Tests do not take a lot of time. They don’t take a lot of money. All they take is a commitment to learning something from your users today.

Pick a Tiny Test that applies to your product and get out and run one right now. Oh, ok. You can wait until you finish the post.

Unmoderated Tests

Dozens of companies now exist that allow you to run an unmoderated test in a few minutes. I’ve used UserTesting.com many times and gotten some great results really quickly. I’ve also heard good things about Loop11 and several others, so feel free to pick the one that you like best.

What you do is come up with a few tasks that you want to see people perform with your product. When the test is over, you get screen captures of people trying to do those things while they narrate the experience.

Typically, I’ll use remote, unmoderated testing when I want to get some quick insight into whether a new feature is usable and obvious for a brand new user.

For example, if you’ve just added the ability for users to message each other on your site, you can use remote, unmoderated testing to watch people attempt to message somebody. This will help you identify the places where they’re getting lost or confused.

If you’ve done a little recruiting and have a list of users who are willing to participate, you can even ask your own users to be the participants.

And don’t forget, if you don’t have a product, or if you’re looking at other products for inspiration, you can run an unmoderated test on a competitor’s product. This can be a great way to see if a particular implementation of a feature is usable without ever having to write a line of code. It can also be a great way to understand where there might be problems with competing products that you can exploit.

Are you going to get as much in-depth, targeted feedback as you would if you ran a really well designed, in person user test? Probably not. But it’ll take you 10 minutes to set up and 15 minutes to watch each video, so you might actually do this.

Remote Observation

There is something to be said for traveling to visit your users and spending time in their homes or offices. It can be extremely educational. It can also be extremely expensive and time consuming.

Here’s a way to get a lot of value with fewer frequent flyer miles.

Look at the people in your Skype contacts. Find one that doesn’t know much about your product. Ping them. Ask them to do three small tasks on your product while sharing their screen.

Don’t have Skype? Send friends a GoToMeeting or a WebEx link through email.

As with the remote unmoderated testing, this is best for figuring out if something is confusing or hard to do. It’s not very useful for figuring out whether people will like or use new features, because typically the people in your Skype contacts aren’t representative of real users of your product.

The closer the people are to your target market, the better the feedback’s going to be, but almost anybody can tell you if something is hard to use, and that’s information that it would be great if you had right now.

Coffee Shop Guerrilla Testing

Of course, it’s tough to test a mobile app over Skype. You know where it’s easy to test a mobile app? At a coffee shop.

Go outside. Find a Starbucks (other coffee shops are also acceptable if you refuse to go to Starbucks, you insufferable snob). Buy some $5 gift cards. Offer to buy people coffee if they spend 5 minutes looking at your product. Have a few tasks in mind that you want them to perform.

In about an hour, you can watch a dozen people use your app. And if you don’t manage to get any good feedback, at least you can get coffee. But you’ll almost certainly get some good feedback.

This type of feedback is great for telling you if a particular task is hard or confusing. It’s also great for getting first impressions of what an app does or the type of person who might use it.

Five Second Landing Page Testing

Sometimes, all you want to test is a new landing page. What you frequently want to know about a landing page is, “What message is this conveying, and is it conveying it clearly and quickly?” Even the tiniest of tests can seem like overkill for that.

For landing pages, I use UsabilityHub’s Five Second Test. You take a screenshot or mockup of the landing page you want to show. You upload it to the site. You enter a few questions you want people to answer after looking at it.

If the whole setup process takes you more than 5 minutes, you’re doing it wrong, and within a few hours you can have dozens of people look at your landing page and tell you what they think your product does.

This sort of Tiny Test is wonderful for testing several different variations of messages or images that you might put on a landing page. You can get people’s real first impressions of what they think you’re trying to tell them.

CTA Testing

The most important thing to get right on any screen is the Call To Action. After all, you can have the most gorgeously designed images with a wonderfully crafted message, but if people can’t find the damn Buy button, you’re screwed.

But, as with the landing page tests, this is something that takes 5 seconds. Basically, you want to show people a screen and see if they can figure out where they should click. Guerrilla testing works pretty well for this, but even that may be overkill here.

For CTA testing, I often use UsabilityHub’s ClickTest product. Again, you just upload a mock and ask people something like, “Where would you click to purchase the product shown on this page?” or “Where would you go to advance to the next slide?” or whatever CTA you’re testing.

A few hours later, you get a map of where people clicked. If there are clicks all over the place, you’ve got some work to do on your CTA.

The advantage to doing something like this over a/b testing is simply that you can get it set up very quickly with just mockups. You don't have to actually implement anything on your site (or even have a site) in order to test this way. But, if you have enough traffic and a good a/b system already set up, by all means test that way, as well.

What Are You Waiting For?

There you go. Five different options for wildly fast, incredibly cheap feedback on your product. You don’t have to hire a recruiter or write a discussion guide or rent out a usability lab. In a few cases, you don’t even have to interact with a human.

Are they perfect? Do they take the place of more substantial research? Will you be able to get away with avoiding talking to your users forever? No. But they’re easy, and you can do one of them right this second.

So...do one of them right this second!

Like the post? Follow me on Twitter.

STFU About What Women Want

In a recent post on TechCrunch, Penelope Trunk tells us (again) that most women don’t want to do startups.

First, I’d like to extend that to Asians, African Americans, Gays, and Latinos. Oh, and white men. Most of them don’t want to do startups either, because most people don’t want to do startups for a whole host of reasons.

Penelope tells us that women are different though, because women don’t want to join startups because women want to have babies. As evidence, she points out that most women downshift their careers as soon as they have babies, which of course makes startups impossible.

It’s not that women don’t join startups because of lack of opportunity or sexism or doing what’s expected of them or anything else. Now that we have completed defeated bias, all women can choose to do anything they want, and they are choosing to have babies rather than go to startups. Case closed!

Here’s the problem: Penelope, and other people who say things like this, are making my life a whole lot harder, and I’d like them to knock it the fuck off.

I’m not going to argue that most women don’t want to stay home with their children. Frankly, I don’t care what most women want to do.

I know what I want to do, and what I want to do is to work at startups. I don’t want to have children. I’ve never wanted children. I never will want children, and I certainly wouldn’t want to give up working at startups for them.

So, when a publication like TechCrunch spews some nonsense about what women want, it means that the next time I go into an interview with a male founder (and they are overwhelmingly male for some reason that I’m not going to address here, but that Penelope assures us has nothing to do with bias) who has read that nonsense, he may be thinking, consciously or subconsciously, “she doesn’t really want to work at this startup because she wants to have a baby.”

And frankly, that sucks for me and all the other women like me. Oh, did I mention that there are lots of other women like me? There are.

But let’s just look for a moment at what all of these other women, the ones with babies and without startups, are choosing to do. They are choosing to stay home because of...I don’t know what the current argument is. Hormones? Biology? Bad government policy? Nature?

It couldn’t possibly be bias or lack of opportunity because, of course, some women are choosing to work at startups, so it would be trivially easy for all women to choose to work at startups, right?

Except that my father’s law school class of 1963 had 3 women in it. That’s right. Three. Now, clearly more women could have joined the class. His law school didn’t have a 3 woman quota or anything.

But the women of 1963 chose not to go to law school. And I’m positive that there were all sorts of blowhards opining that women didn’t go to law school because they were too busy having babies, and this was perfectly normal, and we shouldn’t do a damn thing to promote more women going to law school because WON’T SOMEBODY PLEASE THINK OF THE CHILDREN.

After all, we weren’t actively STOPPING women from going to law school (any longer). It was their choice! Except that, as Penelope points out, more than 50% of the law school graduates now are women.

So, far more women are now choosing to be lawyers. You know, despite the fact that they still have babies. What women wanted, with regard to law school attendance, somehow changed between 1963 and today.

Similarly, long before women had the right to vote in the US, many women didn’t actually want the right to vote. Some even felt that women were biologically not capable of voting well. And for years after they had the right to vote, many people still felt that it was the wrong decision.

How many women in the US do you know who don’t care about voting? Fewer than felt that way a hundred years ago, right?

The point is that “what women want” changes over time. What people want changes over time. Because what we want is hugely driven by social norms and massive cultural shifts and all sorts of things that may seem biological at the time but turn out not to be.

In other words, if suddenly there are a ton of women at startups kicking ass and being awesome, it might turn out that more young women want to join startups in the future. And 25 years from now, we’ll all be laughing at the idiots who said things like “women don’t want to vote...er...go to law school...I mean...join startups!”

Penelope, do you vote? Do you know women who went to law school? I do. And I am forever grateful to the women who fought not just for the right to do these things but to make them seem like totally normal things to do.

I salute the women who said, “Hey, wait a minute. Maybe having a vagina doesn’t determine what I have to want from life! Just because a lot of women want something, doesn’t mean that I have to want the exact same thing!”

And I’m still a little annoyed at the women who said, “Oh, women don’t WANT to vote. Voting is for men!” I take that back. I’m a lot annoyed at them. They sucked.

So stop doing it. Stop assuming other women want to make the same choices you do, especially when society has such an enormous and invisible impact on your choices. Stop assuming a young woman just starting her career knows everything about all of the wonderful, exciting career choices she could make.

And mostly, stop making it ok for other people to assume that I want what you want. That’s clearly not true, since what I want most right at this moment is to punch you in the face.

I am a woman. I want to work at a startup. I don’t want to have children. I want to vote. I want to wear stiletto heels and write jQuery, sometimes at the same time. In other words, I am an individual, and I have all sorts of wants that are neither determined nor predicted by my gender.

I am a woman, Penelope, but you don’t have any idea what I want. So, kindly shut the fuck up about it.

Give the Users What They Really Want

Recently, I’ve been trying to teach startups how to do their own user research. I’ve noticed that I teach a lot of the same things over and over again, since there are a few things about research that seem to be especially difficult for new folks.

One of the most common problems, and possibly the toughest one to overcome, is the tendency to accept solutions from users without understanding the underlying problem. In other words, a user says, “I want x feature,” and instead of learning why they want that feature, new researchers tend to write down, “users want x feature," and then move on.

This is a huge issue with novices performing research, When you do this, you are letting your users design your product for you, and this is bad because, in general, users are terrible at design.

Ooh! An Example!

I participated in some user research for a company with an expensive set of products and services. Users coming to the company’s website were looking for information so they could properly evaluate which set of products and services was right for them. Typically, users ended up buying a custom package of products and services.

One thing we heard from several users was that they really wanted more case studies. Case studies, they said, were extremely helpful.

Now, if you’re conducting user research, and a customer tells you that he wants case studies, this might sound like a great idea.

Unfortunately, the user has just presented you with a solution, not a problem. The reason that this is important is that, based on what the actual underlying problem is, there might be several better solutions available to you.

When we followed up on users’ requests for case studies with the question, “Why do you want to see case studies?” we got a variety of answers. Interestingly, the users asking for case studies were all trying to solve entirely different problems. But were case studies really the best solution for all three problems?

These were the responses along with some analysis.

“I want to know what other companies similar to mine are doing so that I have a good idea of what I should buy.”

The first user’s “problem” was that he didn’t know how to pick the optimal collection of products for his company. This is a choice problem. It’s like when you’re trying to buy a new home theater system, and you have to make a bunch of interrelated decisions about very expensive items that you probably don’t know much about.

While case studies can certainly be helpful in these instances, it’s often more effective to solve choice problems with some sort of recommendation engine or a selection of pre-set packages.

Both of these more quickly help the user understand what the right selection is for him rather than just give him a long explanation of how somebody else found a good solution that might or might not be applicable to the user.

“I want to know what sorts of benefits other companies got from the purchase so I can tell whether it’s worth buying.”

The second user’s “problem” was that he wanted to make sure that he was getting a good value for his money. This is a metrics problem. It’s like when you’re trying to figure out if it’s worth it to buy the more expensive stereo system. You need to understand exactly what you’re getting for your money with each system and then balance the benefits vs the cost.

This problem might have been solved by a price matrix showing exactly what benefits were offered for different products. Alternatively, it would be faster and more effective to display only the pertinent part of the case studies on the product description page - for example, “Customers saw an average of 35% increase in revenue 6 months after installing this product.”

By boiling this down to only the parts of the case study that were actually important to the user, it gives you more flexibility to show this information - statistics, metrics, etc. - in more prominent and pertinent places on the site. This actually increases the impact of these numbers and improves the chance that people will see them.

“I want to see what other sorts of companies you work with so that I can decide whether you have a reputable company.”

The third user’s “problem” was that he hadn’t ever heard of the company selling the products. Since they were expensive products, he wanted the reassurance that companies he had heard of were already clients. This is a social proof problem. It’s like when you’re trying to pick somebody to put a new roof on your house, so you ask your friends for recommendations.

His actual problem could have been solved a lot quicker with a carousel of short client testimonials. Why go to all the trouble of writing up several big case studies when all the user cares about is seeing a Google logo in your client list?

Why This Matters

This shouldn’t come as a surprise to any of you, but users ask for things they’re familiar with, not necessarily what would be best for them. If a user has seen something like case studies before, when he thinks about the value he got from case studies, he’s going to ask for more of the same. He’s not necessarily going to just ask for the part of the case study that was most pertinent to him.

The problem with this is that many people who might also find certain parts of case studies compelling won’t bother to read them because case studies can be quite long or because the user doesn’t think that the particular case study applies to him.

Obviously, this is applicable to a lot more than case studies. For example, I recently saw a very similar situation from buyers and sellers in a social marketplace asking for a “reputation system” when what they really wanted was some sort of reassurance that they wouldn’t get ripped off. I could name a dozen other examples.

The takeaway is that, when somebody asks you for a feature, you need to follow up with questions about why they want the feature, even when you think you already know the answer!

Once you know what their problems really are, you can go about solving them in the most efficient, effective way, rather than the way the user just happened to think of in the study.

Instead of just building what the user asks for, build something that solves the user’s real problem. As an added bonus, you might end up building a smaller, easier feature than the one the user asked for.

The Art of the UX Steal

I’ve been building interfaces for a very long time, and I can tell you that the number of times I’ve had to solve a completely new and unusual user problem is remarkably small. This isn’t surprising. The vast majority of products we build incorporate a lot of familiar elements.

For example, think about the number of products you use that include one or more of the following: login, purchasing, comments, rating systems, order history, inventory management, or user generated content.

Do you expect that every single login experience gets redesigned completely from scratch in a vacuum? Of course not! It would be annoying if they were, since each new version would almost certainly differ just enough to make things confusing. Having design standards for things like logging in makes a lot of sense for both users and designers.

However, this tendency to fall back on patterns, or just to copy whatever Apple/Amazon/Facebook is doing, can cause some problems, especially for startups. There are a few big reasons why you shouldn’t just adopt another company’s solution without serious consideration.

They May Not Want Exactly What You Want

Companies have hidden agendas. But their agenda is not always your agenda, which means that their optimal design is not your optimal design. And if you think that they’re always optimizing for the best user experience, you’ve lost your damn mind.

Want an example? Ok! Have you ever purchased an item and been opted in to receiving email deals from the company’s partner sites? As a user, who likes that? Who thinks that’s a great user experience? Exactly.

Then why do companies do it? They do it because they have made the business (not UX) decision that they make more money by opting people into partner deals than they lose by slightly annoying their customers. That’s a totally reasonable calculation for them to do.

Now, let’s say your biz dev person comes to you and says he wants to add that feature to your checkout process because he has a partner lined up who is willing to pay for the privilege of getting your users’ email addresses. He says it will be ok to add the feature because other big companies are doing it, so it must make money.

But you have no idea how much money they’re getting for making their UX worse. You have no idea of the number of users they may be losing with this practice. And even if you did know their numbers, you can’t decide whether this feature is the right business decision for you until you know what those numbers are going to be for your product.

In an ideal world we could always just choose whatever made the best possible user experience, but realistically, we make these kinds of business/UX tradeoffs all the time. They’re inevitable. Just make sure that you’re making them based on realistic estimates for your product and not on the theory that it’s right because a bigger company is doing it.

They Don’t Do Exactly What You Do

By my count, Amazon has sold at least one of pretty much everything in the world. Ok, I’m just extrapolating from my purchase habits, but you know what I mean.

Not only do they sell products directly, they also allow other companies and individuals to sell through their marketplace. They also sell a lot of different versions of the same product. This makes their product pages pretty complicated.

Does your product do all of those things? If you work for a startup, I certainly hope not, since many of Amazon’s features were added over the course of more than a decade.

If your product doesn’t have all those features, then you might want to do all sorts of things differently than Amazon does. For example, your product pages could be significantly simpler, right? They could emphasize entirely different things or have clearer Calls to Action or more social proof because they don’t need to account for all of Amazon’s different features.

Whether or not you even have product pages, the point is that no other company is doing exactly what you’re doing (or if they are, you have an entirely different problem), so their optimal UX is, by necessity, going to be different from yours.

They Can Get Away with It

If Dante were writing today, the 9th circle of Hell would have involved trying to sign into multiple Google accounts at once. True story.

A friend of mine decided to make me angry the other day, so he showed me a Google docs screen where the Save button was so cleverly hidden it took him several minutes to locate it. This was on a screen that had maybe four elements, and he’s a very senior software engineer, so this probably wasn’t user error. I find the usability on certain Google products almost sadistically poor.

But I put up with it because Google provides me with incredible value for free that I can’t get anywhere else even by paying for it.

I don’t use things like Google docs for their UX. In fact, I use them in spite of large portions of their UX. And if your UX borrows from Google through some misguided notion that just because Google does it, it must be right, I will quit your product in a freaking heartbeat and bad mouth it to all my friends.

The moral of this story isn’t just “don’t steal UX from Google,” although that’s not bad advice. The moral is that very few companies succeed in spite of their UX, and if you happen to steal UX from them, you’re doing it wrong.

On a side note, you know what had a fabulous UX? The original Google product - the one where there was just a single search box, two buttons, and a hugely successful algorithm for finding great results. Unsurprisingly, that’s the UX that got us all hooked in the first place.

The Right Way to Steal

Now that the horror stories are out of the way, you still shouldn’t be coming up with every design element from scratch.

Not only is it ok to steal a basic login process from another product (although not Google), it’s almost certainly the best possible thing you could do. Having a non-standard way for users to log in to your product is just needlessly confusing.

One product I use on a regular basis used to put their Log In button on the top left of their home page instead of the top right. Just this little change meant that several times I had a hard time remembering how to get into the product, and wasted several seconds searching for the button. I probably wasn’t the only one to complain, since they fixed it relatively quickly.

Logging in isn’t the only thing to standardize. Any time you have a simple activity that users do regularly in lots of other products, you should at least check to see whether there is a standard and consider adopting it.

Of course, you can always choose not to do things the way everybody else is doing them, but you should have a very strong reason for changing things, and you should definitely a/b test your change against the standard design pattern.

Trust But Verify

Most importantly, when you are planning on stealing - or “adopting a standard” as we’re now going to euphemistically call it - it’s still important to test it.

I like to do quick qualitative tests to observe some people actually using the standard. In fact, often I’ll test the standard on competitors’ products before implementing it, rather than implementing it and then finding out that it’s crap. Then, I’ll test again once it’s implemented in my product.

In general, the more companies who are doing things identically the less likely it is to be confusing. But it’s still necessary to make sure that the design works in the context of the rest of your product.

Like the post? Follow me on Twitter!

Idiots, Drama Queens, and Scammers - Improving the Customer Service with UX

I recently published another article in Smashing Magazine. This one is titled Idiots, Drama Queens, and Scammers - Improving the Customer Experience with UX.

Here's an excerpt:
User experience design isn’t just about building wireframes and Photoshop mock-ups. It extends to areas that you wouldn’t necessarily think are part of the discipline.

For example, your customer service department can have a huge impact on your website’s overall user experience. Similarly, the design of your user experience could have an awfully big effect on your customer service department. Of course, not all of your users will interact with the customer service department, but for those who do, their experience can improve or destroy the customer relationship.

Read more now >

How Metrics Can Make You a Better Designer

I have another new article in Smashing Magazine's UX section: How Metrics Can Make You a Better Designer.

Here's a little sample:

Metrics can be a touchy subject in design. When I say things like, “Designers should embrace A/B testing” or “Metrics can improve design,” I often hear concerns.

Many designers tell me they feel that metrics displace creativity or create a paint-by-numbers scenario. They don’t want their training and intuition to be overruled by what a chart says a link color should be.

These are valid concerns, if your company thinks it can replace design with metrics. But if you use them correctly, metrics can vastly improve design and make you an even better designer.

Read the rest here >

Need Help with Your Design and Research?

I used to do a lot of design and research for companies. Don't get me wrong. I still do design and research, but I’ve recently made a pretty significant change.

I no longer do design and research FOR companies. I now do design and research WITH companies.

I promise this isn’t just semantic nonsense. It has a huge impact on my relationship with clients, and I think it has some good lessons for people who choose to work with outside UX help.

Give a Company a Fish

Let’s take a look at the typical experience you have when you hire a contractor or an agency. Typically, you give a lot of input to the contractor about what results you want, and the contractor goes off and produces something that hopefully fits those results. 

With a good contractor, you get a lot of discussion and iteration, but at the end, you get a design or a research report that somebody did for you. And that’s all you get.

If you want to change part of the design after the contractor is gone, you run the risk of making major mistakes, because you are very unlikely to understand all the decisions that were made in creating it. If you have a question about the research or want to do a quick follow up about something you learned, you don’t know how to do that yourself.

This means that the next time you want some research or design done, you need to hire somebody to do it for you again. This is great for the contractor, and it’s not bad for companies with big budgets, but it can be especially hard for startups.

Going Fishing Together

Last year, I decided to try a different model. When I was hired by clients, I came in and worked as part of the team. I was still doing the majority of the design and research, but I came in and worked at the office and tried to be integrated into the teams as much as possible.

That worked better than the old agency style I was used to. I had more contact with the engineers and product owners. We could iterate on the design faster because we were all in the same room. I learned far more about the product and users. Sometimes they learned a little about the design process.

Still, with some clients, I found that I was the only person in the room while doing customer research. I was the only one coming up with questions I wanted answered. I was still having to schedule design reviews rather than having everybody involved in the design process.

The worst part was that I was the only one learning anything about the customers. But they weren't MY customers!

Too often, what this meant was, when a project was over, everything at the company went right back to where it was before.

I started to look at why some projects ended this way, while in others, the companies seemed to incorporate good design and research skills into their own development process.

Teach a Company to Fish

Based on what I learned from the companies who improved, I have a different model now for all of my new clients. I’m helping companies learn to do more design and research on their own.

Instead of running a research study, I help product owners figure out what sort of research they need to do. I then help them plan it, execute it, analyze the data, and create actionable designs. If this were a sports team, I’d be a coach, not a ringer.

Of course, this does mean a lot more work for my clients. They have to figure out what questions they want answered. They have to talk to their customers. They have to do design work. They have to understand the process. It’s really hard, and not everybody wants to learn to do these things.

But the beauty of it is, once they’ve done it a few times, it all gets easier. It becomes part of the company process. More people in the company become interested in conducting research and creating designs.

Of course, eventually, my clients won’t need me any longer. It may not be the best business model, but I think it’s the best thing I can do for my clients.

What This Could Mean for You

This means that I can help you learn how to be better at research and design. For example, I can work with you on things like:

• Which type of research is right for you at any given stage of your product development
• How to plan that research correctly
• How to moderate a user discussion properly
• How to analyze your research results
• How to create usable personas and write good user stories
• How to turn research results into actionable designs
• What changes you need to make to your product based on your results
• When to use metrics and a/b testing in your design process
• What to build now, what to test, and what to iterate on later

If you’re interested in any of those things, you should contact me at laura@usersknow.com. I’m happy to discuss the process in more detail and explain a typical engagement.