We’ve said it before, and we’ll say it again: even if you think your website is the most awesome thing since Bill Murray, you’ll never truly know if you’ve got a good thing going until you test it out.
After all, it doesn’t matter how great your content is or how cool your website looks, if people can’t figure out what to do, where to go, or how to navigate through it, they’ll eventually just leave.
To avoid this most painful of all outcomes, be sure to test your website with actual users before you go live. This will help you to identify both the obvious and not-so-obvious flaws so you can refine the experience into one that has the greatest potential to convert. Here are a couple of things to consider and a checklist to help you plan.
User testing, usability testing, and A|B testing are totally different beasts.
News flash: user testing and usability testing are different things. User testing (aka focus groups) helps to gauge user aesthetic preferences, expectations, emotional responses, and other opinion-based metrics. Usability, on the other hand, is all about how a user performs specific tasks and understands website functionality or organization; how your website makes someone feel is important, but it has little do with how usable your site is.
And then there’s A|B testing. Firstly, let us just say that we love A|B testing. In fact, if this were kindergarten, we would marry it if we could. There’s pretty much no better way to find out what design elements, content, and copy will resonate most with users and encourage them to take action/convert. The thing is, an A|B test is about optimizing a site that already has some baseline of performance. It involves creating slightly different web pages with different elements to test, and requires a lot of traffic to achieve any kind of statistical significance that can influence change.
A|B testing is a great way to figure out what’s working and what isn’t—what it doesn’t tell you is the why, how, and what behind what’s working and what isn’t. For that, you need insights provided by user and usability testing, and those things should happen before you nudge your website out of the nest and into the world.
While all three types of testing are important, it is usability testing first and foremost that helps facilitate the others. So how does one get started? It’s not always easy, but the good folks at Usability.gov have provided a comprehensive usability testing planning guide which you can read here. We’ve taken what we consider the most essential steps and included them below:
1. Define your scope + purpose.
You can’t just send people to your website and tell them to have at it. You’ll first need to define which parts of your website you want to test. The whole thing? Just the homepage? Just the copy or content? Then you’ll need to narrow it down by purpose of the test. For example, how difficult is it for people to locate the search bar? Asking participants to accomplish this task and rate the difficulty will help you test and refine your site’s navigation design.
2. Determine the time, place, and length of sessions.
Usability testing is a one-on-one affair, so you’ll need to make sure you’ve got enough time and resources to accommodate participants and learn as much as you can. Remember to leave enough time after each session for Q&A and to reset the test for subsequent participants.
3. Get your gear in order.
Will you test your website’s usability on desktop, tablet, or smartphone? Figure out what equipment you’re going to need to accurately recreate testing scenarios against how you anticipate people will interface with your site in the real world. Ideally—and in consideration of how people consume content nowadays (mobile accounts for one-third of all global website traffic!)—you’ll want all three.
4. Round up some participants.
Now’s the time to figure out how many testers are right for what you’re trying to accomplish, and who they are. Ideally, participants should match your target demographic so their results will reflect the optimal user experience. As for number of testers, remember that whole thing about how A|B testing takes a lot of users while usability testing…doesn’t? This study makes a compelling case for testing with just 5 users—that is, 5 users per iterative round (15 total). The theory is that, by the fifth tester, you stop learning new insights. Instead, take what you’ve learned from the first five, and test again with a new crew of five, and so on; it’s a more efficient use of your time and budget.
5. Identify what’s worth measuring.
Every test you create should center around the participant accomplishing a specific task, and the metrics should represent various aspects of that. For example, you’ll not only want to record if the task was successfully completed, but also how long it took them to do so. Were there any stumbling blocks that caused task completion to be less efficient? How many participants were able to complete tasks without errors? These metrics will help you to tweak and refine after each testing group.
Usability testing can be a time-consuming and (often) humbling process, but it’s the best way to give your website (and, by association, your brand!) a foundation for success out there in the Internet wilderness. That’s why it’s so important to work both usability and user testing in before you launch, and on a regular basis as your products, offerings, or brand, changes—it’s a relatively small investment that can save you a lot of money and grief in the long run.