A lot of people recently have asked me about the new crop of user testing tools available on the internet. One specific tool that comes up a lot is usertesting.com, and I’d like to talk a little bit about my experience with it.
Frankly, when I first saw the site, my initial reaction was, “Well, that’s not going to be very useful.” I’ve heard a similar gut reaction from several user researchers. Once I’d given it a try, my reaction changed to, “Oh, shit. This could seriously cut into my income.”
Having used it several times now, I can happily say that neither of these reactions was correct.
Having used it several times now, I can happily say that neither of these reactions was correct.
The Cons:
Let’s start with the things that I originally noted about usertesting.com that made me think it wouldn’t be very useful.
There is no moderator.
Not having a moderator means that there isn’t a human being running the test who can ask follow up questions and delve deeply into issues that come up naturally during a session. Good moderators don’t just follow a script; they ask the right questions to really understand why users are doing what they’re doing.
It’s less useful for testing incomplete interactive prototypes.
I have not yet found a good way to test prototypes with usertesting.com. When I design, I create very sketchy, incomplete mockups of products. Typically, these only run in one browser, they have no visual design, and large parts of them won’t really work or will use fake data.
I can deal with all of these issues in a one-on-one session by explaining that the prototypes are not real, giving the participant some help in areas where the prototype isn’t perfect, or gently reminding them not to fixate on the visual design. This isn’t really possible without a human moderator.
