What Makes UX Lean - My Talk from SXSW

If you couldn’t make it to SXSW this year, there was a fantastic, all day lean startup track with talks from lots of lean startup experts. I was lucky enough to be asked to be on the Lean UX panel, along with the always awesome Janice Fraser, Ian McFarland, and Dan Martell.

I gave a short talk on what makes Lean UX Lean. Since I’m a blogger at heart, I wrote down pretty much everything I was going to say first, which means I can now publish a draft of the talk here! If you didn’t get to hear the panel, or if you did and want a quick refresher, please enjoy!

I’ve been a user experience designer for a lot of years, and I’ve worked with a lot of lean startups, which is part of the reason why I got a call last year from Manuel Rosso, the CEO of Food on the Table.

Now, Food on the Table is a very lean startup here in Austin. Because they’re a lean startup, they measure absolutely everything. And because they measure everything, Manuel knew immediately when the product developed an activation problem.

The whole project has been written up in a post for Eric’s Startup Lessons Learned blog, and I strongly recommend that you go read it if you haven’t already. It has a lot of tips about how to incorporate design into your startup that you’ll hopefully find helpful.

But today, I want to go a little deeper into what made that project a good example of Lean UX. Because, during that project, we did a lot of things that you might do in any sort of a UX project for any sort of a company.

For example, we did qualitative user research to understand why users were having a problem. We made sketches and built interactive prototypes, and we tested and iterated on them.

These are wonderful, helpful things to do, but they’re not unique to Lean User Experience Design. They’re part of User Centered Design, which I’m a huge fan of, and I’ve done all of those things in waterfall projects at giant companies that were anything but lean.

So, what are a few things that made this a lean ux project and not just a regular old redesign?

Integrating Quantitative Research

I think the first hallmark of Lean UX is using quantitative metrics to both drive and validate design changes. What does that mean? Well, it means that the reason we were working on the first time user experience was because a specific metric, activation, wasn’t as high as the team wanted it to be.

Quantitative metrics didn’t tell us exactly why we had a problem - we needed to do our qualitative research to understand that - but it did tell us what our most immediate problem was, which helped us to understand where we should start improving our user experience.

In that way, the metric drove the product decision.

Quantitative metrics also meant that we knew, at the end of the project, we’d be validating our work with an a/b test against the original design. That quantitative validation of design really helps improve the design process over the long run, because we can see what sorts of changes have the biggest positive impact on our end user experience. That lets us improve the ROI on future design projects.

When Is a Design Done?

I was talking with a designer about Lean UX. I was explaining that one of the hallmarks of Lean UX is to get a good, but not complete version of a product or feature designed and built and then iterate on it later. She thought this sounded like an interesting approach, but then she asked, “When do you know you’re done?”

Figuring out when you’re “done” is tricky for any design or redesign project, unless you’re a consulting agency, of course, in which case the answer is, “when the client runs out of money.” But I realized that, in Lean UX, figuring out when you’re done is actually incredibly easy.

You’re done when your metrics tell you you’re done.

Let me explain. No product is ever actually “done.” There is always something you could do to improve it. However, projects can certainly be done. The trick is that you have to choose your projects correctly.

What’s the correct way to choose a Lean UX project? Every Lean UX project should be chosen based on a metric.

This may piss off a lot of designers who want to make wonderful, exciting, super cool designs just for the sake of design or user happiness, but when it comes down to it, unless you’re independently wealthy, every design change you make should move a number that is important to your business.

Now, it is a lucky break for those of us who care deeply about our users that improving the overall user experience of the product frequently improves some number that the business people care about. But not every single thing you can do to make a user happy has the same ROI for the business. And not every improvement makes the right people happy at the right time.

That’s why the UX projects you choose should be based on metrics.

Let me give you an example. Whoever it is at your startup who is in charge of running the business should have a pretty good idea of what your various metrics have to be in order for you to all retire and buy yachts. For example, your Activation number may have to be 20% and your Retention may have to be 70%. (Please note, I made these numbers up. Your metrics may vary.)

They pick these numbers because they know that having, for example, a 99% retention rate and a 1% activation rate may lead to retaining 3 incredibly happy users forever, which is suboptimal from a business perspective.

So, if your activation number is at 10%, your business folks may come to you and say, “we need to turn more of our acquired traffic into regular users because we have identified this as the most important problem to solve at this moment.” You respond, “Great! How many more do you need?” They explain that you need to get activation from 10% to 20%.

You will notice that the metrics are not driving your design decisions. Nor are they driving your feature requirements or any other product changes. They are simply telling you what your biggest business problem currently is.

Now, it’s up to you as a designer or product owner to figure out what is keeping the activation number low and then come up with some ideas of how to fix it. You do this with what I like to call “research and design” or alternately “that thing you are paid to do.”

You may have dozens of wonderful ideas for how to fix the problem, and you may love and believe in all of them. You may not, however, actually execute every single one of them.

This is where the Lean part comes in.

Ideally, you will design and execute as many of the fixes as necessary in order to move the number to where you want it to be. Maybe you’re awesome (or awesomely lucky), and you move that activation number on the first try with a very small bug fix.

Does that mean you never get to implement the super sweet, but somewhat complicated, feature that you know will make users incredibly happy and improve activation even more? No! Unfortunately, you may not get to implement it just yet.

You see, once you got your activation number to where it needed to be, it stopped being the most important problem to solve. Now, maybe you need to work on getting retention higher or improving revenue or referral.

On the flip side, maybe you redesign the first time user flow and improve activation, but not by enough. That means you should continue working on it. Figure out why your changes didn’t have as big of an impact as you thought they would, and then try some new things.

You’re not “finished” until your metrics are where you want them to be.

Why is this important? Startups have a ridiculous number of things to do, and they typically have limited resources. It can be incredibly difficult to prioritize when to keep working on a feature or an area of the product, and when to move on.

By setting the goals ahead of time based on metrics that are critical to the business, it becomes much easier to know when you’re “done,” and when you should keep optimizing or redesigning.

Like the post? Follow me on Twitter!

Like Lean UX but hate reading? I'll be on the UX panel at the Lean Startup track at SXSW. You should come see it and then say hi to me afterward.