The danger of hidden usability issues

If you have worked in the web development industry, you’ve probably had the following scenario play out on more than one of your projects.

You work on something for a year. Your internal team tests extensively for months. You are proud and excited about the work you’ve done. You launch the site to the public and within days real users identify a variety of small bugs and usability problems with your project. Despair and panic ensues.

We recently had this happen to us. We launched a high profile new site for a client that featured a unique and innovative user interface. Within days of launch it became clear that a small percentage of users didn’t understand how to use aspects of the site. An even smaller percentage were using obscure mouse configurations that prevented them from being able to access key site features.

We were fortunate – the problems that were identified post launch by real users were relatively minor. With just a few days of additional work we were able to make improvements that addressed the issues our users reported. The site is performing really well and overall feedback has been extremely positive.

However, it obviously would have been better to identify and fix these problems prior to launching the site to the general public.

Testing with real users

Small usability problems and obscure bugs are really hard to catch, largely because the traditional quality assurance process has some pretty big blind spots. No matter how many different browsers and devices you test on, ultimately the people that test during the web development process are not usually real users.

Quality control on sites is almost always done by web development professionals who have an innate understanding of user interfaces. Often, the people testing have been looking at the site for months and know it in and out. The way these testers use and view the site is fundamentally different from the way a real customer or constituent would use it.

If you have the time and budget, the best way to test your site for usability problems is to have real people use it before releasing it to the world. There are a few different approaches we have taken to doing this:

• Recruit testers from your user base. Perhaps the simplest way to conduct usability testing is to send your site to a group of users prior to launch and ask for feedback. Send the site to a small percentage of your email list and ask them to provide feedback. Or recruit a panel of testers using a tool like www.usertesting.com.
• Test your site on live traffic. Time and budget permitting, the best way to test your site is by sending live traffic to it. Direct a small percentage (10-25%) of site traffic to your new site and collect quantitative and qualitative feedback on how it performs. Make adjustments and tweaks based on the data you collect and only launch when you are confident your new site is bug free and will outperform your old one.

Launching a new site is an exciting and scary process. Testing your site on real users is a great way to take some of the stress out of the process and minimize the blind spots in the traditional web development process.