Tiny Tests: User Research You Can Do NOW!

There’s a lot of advice about how to do great user research. I have some pretty strong opinions about it myself.

But, as with exercise, the best kind of research is the kind that you actually DO.

So, in the interests of getting some good feedback from your users right now, I have some suggestions for Tiny Tests. These are types of research that you could do right this second with very little preparation on your part.

What Is a Tiny Test?

Tiny Tests do not take a lot of time. They don’t take a lot of money. All they take is a commitment to learning something from your users today.

Pick a Tiny Test that applies to your product and get out and run one right now. Oh, ok. You can wait until you finish the post.

Unmoderated Tests

Dozens of companies now exist that allow you to run an unmoderated test in a few minutes. I’ve used UserTesting.com many times and gotten some great results really quickly. I’ve also heard good things about Loop11 and several others, so feel free to pick the one that you like best.

What you do is come up with a few tasks that you want to see people perform with your product. When the test is over, you get screen captures of people trying to do those things while they narrate the experience.

Typically, I’ll use remote, unmoderated testing when I want to get some quick insight into whether a new feature is usable and obvious for a brand new user.

For example, if you’ve just added the ability for users to message each other on your site, you can use remote, unmoderated testing to watch people attempt to message somebody. This will help you identify the places where they’re getting lost or confused.

If you’ve done a little recruiting and have a list of users who are willing to participate, you can even ask your own users to be the participants.

And don’t forget, if you don’t have a product, or if you’re looking at other products for inspiration, you can run an unmoderated test on a competitor’s product. This can be a great way to see if a particular implementation of a feature is usable without ever having to write a line of code. It can also be a great way to understand where there might be problems with competing products that you can exploit.

Are you going to get as much in-depth, targeted feedback as you would if you ran a really well designed, in person user test? Probably not. But it’ll take you 10 minutes to set up and 15 minutes to watch each video, so you might actually do this.

Remote Observation

There is something to be said for traveling to visit your users and spending time in their homes or offices. It can be extremely educational. It can also be extremely expensive and time consuming.

Here’s a way to get a lot of value with fewer frequent flyer miles.

Look at the people in your Skype contacts. Find one that doesn’t know much about your product. Ping them. Ask them to do three small tasks on your product while sharing their screen.

Don’t have Skype? Send friends a GoToMeeting or a WebEx link through email.

As with the remote unmoderated testing, this is best for figuring out if something is confusing or hard to do. It’s not very useful for figuring out whether people will like or use new features, because typically the people in your Skype contacts aren’t representative of real users of your product.

The closer the people are to your target market, the better the feedback’s going to be, but almost anybody can tell you if something is hard to use, and that’s information that it would be great if you had right now.

Coffee Shop Guerrilla Testing

Of course, it’s tough to test a mobile app over Skype. You know where it’s easy to test a mobile app? At a coffee shop.

Go outside. Find a Starbucks (other coffee shops are also acceptable if you refuse to go to Starbucks, you insufferable snob). Buy some $5 gift cards. Offer to buy people coffee if they spend 5 minutes looking at your product. Have a few tasks in mind that you want them to perform.

In about an hour, you can watch a dozen people use your app. And if you don’t manage to get any good feedback, at least you can get coffee. But you’ll almost certainly get some good feedback.

This type of feedback is great for telling you if a particular task is hard or confusing. It’s also great for getting first impressions of what an app does or the type of person who might use it.

Five Second Landing Page Testing

Sometimes, all you want to test is a new landing page. What you frequently want to know about a landing page is, “What message is this conveying, and is it conveying it clearly and quickly?” Even the tiniest of tests can seem like overkill for that.

For landing pages, I use UsabilityHub’s Five Second Test. You take a screenshot or mockup of the landing page you want to show. You upload it to the site. You enter a few questions you want people to answer after looking at it.

If the whole setup process takes you more than 5 minutes, you’re doing it wrong, and within a few hours you can have dozens of people look at your landing page and tell you what they think your product does.

This sort of Tiny Test is wonderful for testing several different variations of messages or images that you might put on a landing page. You can get people’s real first impressions of what they think you’re trying to tell them.

CTA Testing

The most important thing to get right on any screen is the Call To Action. After all, you can have the most gorgeously designed images with a wonderfully crafted message, but if people can’t find the damn Buy button, you’re screwed.

But, as with the landing page tests, this is something that takes 5 seconds. Basically, you want to show people a screen and see if they can figure out where they should click. Guerrilla testing works pretty well for this, but even that may be overkill here.

For CTA testing, I often use UsabilityHub’s ClickTest product. Again, you just upload a mock and ask people something like, “Where would you click to purchase the product shown on this page?” or “Where would you go to advance to the next slide?” or whatever CTA you’re testing.

A few hours later, you get a map of where people clicked. If there are clicks all over the place, you’ve got some work to do on your CTA.

The advantage to doing something like this over a/b testing is simply that you can get it set up very quickly with just mockups. You don't have to actually implement anything on your site (or even have a site) in order to test this way. But, if you have enough traffic and a good a/b system already set up, by all means test that way, as well.

What Are You Waiting For?

There you go. Five different options for wildly fast, incredibly cheap feedback on your product. You don’t have to hire a recruiter or write a discussion guide or rent out a usability lab. In a few cases, you don’t even have to interact with a human.

Are they perfect? Do they take the place of more substantial research? Will you be able to get away with avoiding talking to your users forever? No. But they’re easy, and you can do one of them right this second.

So...do one of them right this second!

Like the post? Follow me on Twitter.

STFU About What Women Want

In a recent post on TechCrunch, Penelope Trunk tells us (again) that most women don’t want to do startups.

First, I’d like to extend that to Asians, African Americans, Gays, and Latinos. Oh, and white men. Most of them don’t want to do startups either, because most people don’t want to do startups for a whole host of reasons.

Penelope tells us that women are different though, because women don’t want to join startups because women want to have babies. As evidence, she points out that most women downshift their careers as soon as they have babies, which of course makes startups impossible.

It’s not that women don’t join startups because of lack of opportunity or sexism or doing what’s expected of them or anything else. Now that we have completed defeated bias, all women can choose to do anything they want, and they are choosing to have babies rather than go to startups. Case closed!

Here’s the problem: Penelope, and other people who say things like this, are making my life a whole lot harder, and I’d like them to knock it the fuck off.

I’m not going to argue that most women don’t want to stay home with their children. Frankly, I don’t care what most women want to do.

I know what I want to do, and what I want to do is to work at startups. I don’t want to have children. I’ve never wanted children. I never will want children, and I certainly wouldn’t want to give up working at startups for them.

So, when a publication like TechCrunch spews some nonsense about what women want, it means that the next time I go into an interview with a male founder (and they are overwhelmingly male for some reason that I’m not going to address here, but that Penelope assures us has nothing to do with bias) who has read that nonsense, he may be thinking, consciously or subconsciously, “she doesn’t really want to work at this startup because she wants to have a baby.”

And frankly, that sucks for me and all the other women like me. Oh, did I mention that there are lots of other women like me? There are.

But let’s just look for a moment at what all of these other women, the ones with babies and without startups, are choosing to do. They are choosing to stay home because of...I don’t know what the current argument is. Hormones? Biology? Bad government policy? Nature?

It couldn’t possibly be bias or lack of opportunity because, of course, some women are choosing to work at startups, so it would be trivially easy for all women to choose to work at startups, right?

Except that my father’s law school class of 1963 had 3 women in it. That’s right. Three. Now, clearly more women could have joined the class. His law school didn’t have a 3 woman quota or anything.

But the women of 1963 chose not to go to law school. And I’m positive that there were all sorts of blowhards opining that women didn’t go to law school because they were too busy having babies, and this was perfectly normal, and we shouldn’t do a damn thing to promote more women going to law school because WON’T SOMEBODY PLEASE THINK OF THE CHILDREN.

After all, we weren’t actively STOPPING women from going to law school (any longer). It was their choice! Except that, as Penelope points out, more than 50% of the law school graduates now are women.

So, far more women are now choosing to be lawyers. You know, despite the fact that they still have babies. What women wanted, with regard to law school attendance, somehow changed between 1963 and today.

Similarly, long before women had the right to vote in the US, many women didn’t actually want the right to vote. Some even felt that women were biologically not capable of voting well. And for years after they had the right to vote, many people still felt that it was the wrong decision.

How many women in the US do you know who don’t care about voting? Fewer than felt that way a hundred years ago, right?

The point is that “what women want” changes over time. What people want changes over time. Because what we want is hugely driven by social norms and massive cultural shifts and all sorts of things that may seem biological at the time but turn out not to be.

In other words, if suddenly there are a ton of women at startups kicking ass and being awesome, it might turn out that more young women want to join startups in the future. And 25 years from now, we’ll all be laughing at the idiots who said things like “women don’t want to vote...er...go to law school...I mean...join startups!”

Penelope, do you vote? Do you know women who went to law school? I do. And I am forever grateful to the women who fought not just for the right to do these things but to make them seem like totally normal things to do.

I salute the women who said, “Hey, wait a minute. Maybe having a vagina doesn’t determine what I have to want from life! Just because a lot of women want something, doesn’t mean that I have to want the exact same thing!”

And I’m still a little annoyed at the women who said, “Oh, women don’t WANT to vote. Voting is for men!” I take that back. I’m a lot annoyed at them. They sucked.

So stop doing it. Stop assuming other women want to make the same choices you do, especially when society has such an enormous and invisible impact on your choices. Stop assuming a young woman just starting her career knows everything about all of the wonderful, exciting career choices she could make.

And mostly, stop making it ok for other people to assume that I want what you want. That’s clearly not true, since what I want most right at this moment is to punch you in the face.

I am a woman. I want to work at a startup. I don’t want to have children. I want to vote. I want to wear stiletto heels and write jQuery, sometimes at the same time. In other words, I am an individual, and I have all sorts of wants that are neither determined nor predicted by my gender.

I am a woman, Penelope, but you don’t have any idea what I want. So, kindly shut the fuck up about it.

Give the Users What They Really Want

Recently, I’ve been trying to teach startups how to do their own user research. I’ve noticed that I teach a lot of the same things over and over again, since there are a few things about research that seem to be especially difficult for new folks.

One of the most common problems, and possibly the toughest one to overcome, is the tendency to accept solutions from users without understanding the underlying problem. In other words, a user says, “I want x feature,” and instead of learning why they want that feature, new researchers tend to write down, “users want x feature," and then move on.

This is a huge issue with novices performing research, When you do this, you are letting your users design your product for you, and this is bad because, in general, users are terrible at design.

Ooh! An Example!

I participated in some user research for a company with an expensive set of products and services. Users coming to the company’s website were looking for information so they could properly evaluate which set of products and services was right for them. Typically, users ended up buying a custom package of products and services.

One thing we heard from several users was that they really wanted more case studies. Case studies, they said, were extremely helpful.

Now, if you’re conducting user research, and a customer tells you that he wants case studies, this might sound like a great idea.

Unfortunately, the user has just presented you with a solution, not a problem. The reason that this is important is that, based on what the actual underlying problem is, there might be several better solutions available to you.

When we followed up on users’ requests for case studies with the question, “Why do you want to see case studies?” we got a variety of answers. Interestingly, the users asking for case studies were all trying to solve entirely different problems. But were case studies really the best solution for all three problems?

These were the responses along with some analysis.

“I want to know what other companies similar to mine are doing so that I have a good idea of what I should buy.”

The first user’s “problem” was that he didn’t know how to pick the optimal collection of products for his company. This is a choice problem. It’s like when you’re trying to buy a new home theater system, and you have to make a bunch of interrelated decisions about very expensive items that you probably don’t know much about.

While case studies can certainly be helpful in these instances, it’s often more effective to solve choice problems with some sort of recommendation engine or a selection of pre-set packages.

Both of these more quickly help the user understand what the right selection is for him rather than just give him a long explanation of how somebody else found a good solution that might or might not be applicable to the user.

“I want to know what sorts of benefits other companies got from the purchase so I can tell whether it’s worth buying.”

The second user’s “problem” was that he wanted to make sure that he was getting a good value for his money. This is a metrics problem. It’s like when you’re trying to figure out if it’s worth it to buy the more expensive stereo system. You need to understand exactly what you’re getting for your money with each system and then balance the benefits vs the cost.

This problem might have been solved by a price matrix showing exactly what benefits were offered for different products. Alternatively, it would be faster and more effective to display only the pertinent part of the case studies on the product description page - for example, “Customers saw an average of 35% increase in revenue 6 months after installing this product.”

By boiling this down to only the parts of the case study that were actually important to the user, it gives you more flexibility to show this information - statistics, metrics, etc. - in more prominent and pertinent places on the site. This actually increases the impact of these numbers and improves the chance that people will see them.

“I want to see what other sorts of companies you work with so that I can decide whether you have a reputable company.”

The third user’s “problem” was that he hadn’t ever heard of the company selling the products. Since they were expensive products, he wanted the reassurance that companies he had heard of were already clients. This is a social proof problem. It’s like when you’re trying to pick somebody to put a new roof on your house, so you ask your friends for recommendations.

His actual problem could have been solved a lot quicker with a carousel of short client testimonials. Why go to all the trouble of writing up several big case studies when all the user cares about is seeing a Google logo in your client list?

Why This Matters

This shouldn’t come as a surprise to any of you, but users ask for things they’re familiar with, not necessarily what would be best for them. If a user has seen something like case studies before, when he thinks about the value he got from case studies, he’s going to ask for more of the same. He’s not necessarily going to just ask for the part of the case study that was most pertinent to him.

The problem with this is that many people who might also find certain parts of case studies compelling won’t bother to read them because case studies can be quite long or because the user doesn’t think that the particular case study applies to him.

Obviously, this is applicable to a lot more than case studies. For example, I recently saw a very similar situation from buyers and sellers in a social marketplace asking for a “reputation system” when what they really wanted was some sort of reassurance that they wouldn’t get ripped off. I could name a dozen other examples.

The takeaway is that, when somebody asks you for a feature, you need to follow up with questions about why they want the feature, even when you think you already know the answer!

Once you know what their problems really are, you can go about solving them in the most efficient, effective way, rather than the way the user just happened to think of in the study.

Instead of just building what the user asks for, build something that solves the user’s real problem. As an added bonus, you might end up building a smaller, easier feature than the one the user asked for.