Pages

▼

Monday, April 19, 2010

6 Questions To Ask Before Launching a New Feature

There are a lot of things to pay attention to when you’re launching a new feature. This post is going to cover a few of the questions that you really ought to ask before launch, but may not have in the past. Now, to be clear, these are not the obvious questions like:
  • Is this addressing a user need or pain point?
  • Has this been through and passed some form of QA testing?
  • Do I have a marketing plan for letting users know this exists?
  • What is the expected ROI for this feature?
  • Will this feature scale to a reasonable number of users?
  • Is this feature usable, and is the UX reasonably consistent with the rest of my product?
If you have not asked the above questions about the feature you’re about to release, then you should stop reading this blog post and answer them right now.

The questions we’re asking today are more advanced, but your launch will go a lot more smoothly if you ask (and answer!) them ahead of time.

How Am I Going to Get Feedback on this Feature?

It’s not enough to just launch a feature and see if your revenue increases. You need to have some sort of plan for testing to see how it’s affecting key metrics, whether those are revenue, retention, registration, user happiness, or some other number you care about. You need to have a strategy for finding how users feel about the new feature, whether they’re using it, and how it’s affecting your bottom line.

A good answer to this question might include some or all of the following:
  • I am initially releasing this feature to a small cohort of random users and A/B testing key metrics of the sample group vs. the control.
  • I have embedded a short survey or feedback link into the page and will gather qualitative data from my users that way.
  • I have contacted my customer service/community management team and prepped them with information about the new feature and asked them to get back to me if we start to see any significant new support requests.
  • I have already released the feature to a small, select group of beta customers or my customer advisory board, and I am meeting with them to discuss their feedback.

Can I Change this Feature Based on the Feedback I Get?

If your feature is absolutely perfect the first time you release it, either you’re incredibly lucky, or you didn’t release it early enough. Let's just say, that's not usually the case, so just because your feature is released doesn’t mean you don’t need to put it on the schedule for the next sprint!  Please plan for iteration.

A good answer to this question should really include all of the following:
  • I have both development and design time scheduled for quick follow up work on the feature over the next few sprints.
  • My code base can be changed or refactored to support major changes to the feature.

Do I Have the Time and Willingness to Support and Improve this Feature Going Forward?

Your dedication to this new feature needs to extend past immediate iterations. Remember, if significant numbers of users adopt this feature, you could end up supporting it for a very long time. The more features your overall product has, the harder it can be to support all of them well and the more complicated your product can get, so this is not a question that should be treated lightly.

The answer to this question must be:
  • I have enough design, engineering, QA, and customer support resources to dedicate to supporting and improving this feature without causing me to neglect any of the other important features that I’m already supporting. This is not just true for the upcoming sprint, but for the entire future of this feature.

What’s My Plan for Killing the Feature if Things Go Wrong?

There are a lot of reasons to kill a feature. Your users hate it or don’t adopt it at a high enough rate to justify supporting it. It causes massive scalability or performance problems. It doesn’t fit in with the rest of your offering. The ROI isn’t good enough. Your UX has become too cluttered, and you need to pare down your feature list.

You must be prepared to kill features, and it’s always easiest to kill a feature in the earliest stages before it develops a small but vocal group of devoted users. It’s also easier to kill a feature when you have a plan for doing it.


The answer to this question should include several of the following:
  • I am closely monitoring the feature to let me know if it needs to be killed for performance or scalability reasons.
  • I have an automatic rollback system in place that will shut down the rollout of this feature if there are obvious and immediate performance problems.
  • I have a kill switch on the entire feature that will allow me to shut it down manually or limit the damage if performance or scalability problems threaten the system later.
  • I have clearly labeled the feature as “beta” or “experimental” to let users know that it is being evaluated, and I have provided a clear warning that it is not necessarily permanent.
  • I am rolling the feature out to a small group of users first, so that, if I have to kill it, it will not affect my entire user base.
  • I am not charging customers for it yet, since I know that it is much harder to pull a feature that customers have paid for than one that is provided for free. Or, if I am charging, I have a plan in place for how to compensate users when it is killed.
  • I have alerted my customer service or community management staff so that they can prepare a consistent message to my users if the feature must be killed.

Is This Feature Likely to Annoy Some of My Users?

Obviously, it would be lovely if all of your products features thrilled all of your users. Still, sometimes you know that a certain group of people are likely to be antagonized by a particular change, and you decide to make it anyway. That’s a completely valid business decision, but you need to be prepared for the reaction and ready to contain the damage. As a note, this applies equally to killing features, so feel free to ask yourself this question before you pull the plug.

Appropriate answers to this question may include:
  • It may anger an acceptable percentage of my users, but my customer support and community management teams are fully prepared with a good message and possibly some sort of compensation for unhappy users.
  • It is possible that it will anger enough of my users that I will have to implement my well prepared plan to kill the feature.
  • It is extremely unlikely to annoy anybody, but I have still briefed my customer contact points that a new feature is being released so that they can immediately inform me in the event of unexpected problems.
  • I have already reached out to the group of users that this feature is most likely to antagonize and included them in the design process, so they feel more invested in the feature and are less likely to complain.

Do I Have a Plan to Monetize it?

I toyed with putting this one into the list of really basic questions that nobody would need reminding of, but then I realized that I was being hopelessly optimistic. Look, I don’t actually think that every single feature needs to make direct revenue. I’m pretty sure that just making your customers happier leads to more revenue if you have any sort of monetization model in place. That, unfortunately, is a big “if,” so make sure that you’re at least asking the question about whether you should be monetizing a feature before you ship it.

Changing your monetization strategy after the fact can be quite difficult. Suddenly and without warning charging for something you’ve been giving away for free can anger your customers. Deciding to give away something you’ve been charging for can make customers who paid for it feel ripped off. And, once you’ve made something available to the general public, it can be impossible to then make it exclusive. So, at least have a plan for what you’re going to do with this thing in terms of making money off of it, ok?

Appropriate answers to this question could be along the lines of:
  • While this feature will not earn direct revenue, it is expected to increase registration/retention/happiness and will cause users to sign up for the service or remain subscribers/paying customers.
  • This feature will be part of a premium package that will only be available to subscribers/VIP members/paying customers.
  • This feature will be an add-on that users will have to pay to access.
  • This feature will be free while it’s in beta, but there is an end date at which we will begin charging, and users are made aware of this fact before they start using it.
  • It will be very clearly stated that this feature will be available to everybody for a trial period, but they will have to pay to continue using it.
  • The basic version of this feature will be available to everybody, but an upgraded version will cost money.

My God, How Long Will This Take?

I know it seems like a lot to do, but really, a lot of these questions can be handled once for your whole product. For example, putting in a system to kill or roll back dangerous features can be done upfront and used for all product improvements.

Besides, each individual question shouldn’t take that long to answer. And trust me, however long they do take, answering these questions now will save you a huge amount of time in the future.

Like this post?  Do some of the following:
at