Using great visuals = Failing your customers

A week ago we brought you three reasons why A/B testing fails in web design, including but not limited to:

1. The 30 percent rule.
2. Designers do not automatically know what clients crave.
3. The US is not the world – LOL (laughing out loud).

Below we share how this was applied to one of our own websites and outline the lessons we came away with from this exercise. To give you the biggest benefit, please download the 8-page document:

Download – ComMetrics screenshots – annotations about how the design changed through A/B testing

Step 1: Yes, creating a mock-up helps
As shown in the above document, we made a mock-up based on what we knew about usability design, our server-side statistics, guideline reviews and input by other experts, in the hopes that it would suit our clients.

It was then submitted to some reviewers and was tweaked after we looked through some sites we thought did a great job. Of course, the question was whether our clients would like it.

Lesson 1: Selling legal services in Brazil is not the same as in Canada, and building a website for books in Germany or designer coffee in Sweden are completely different ballgames. The reality check is what your customers need and prefer. If your mock-up design does not reflect this, you are off to a bad start.

Step 2: Choosing the designer(s) is the easy part
We wanted a team that successfully uses social media and networks for their own purposes, while going beyond the usual fare.

Most designers have a preference and tend to repeat themselves over and over with each new contract – so the real clincher is whether you can get them to think outside the box.

Lesson 2: You must be able to convince the designer(s) that meeting your clients’ usability needs is what matters, not their preferences for certain approaches.

Step 3: User feedback is key
We started with two choices and did an A/B test by showing users two designs and asking them their preference. We took their responses regarding choices 1 and 2, as well as some additional comments to create another design (see page 4 in the above document).

Lesson 3: Convincing designers that things may be quite the opposite of their own preferences is tricky. However, smart designers will always consider client preferences (meaning your customers’ usability needs) a valid reason to change things.

Step 4: Keep some testers from beginning to end
We also added new testers in every new review cycle while keeping some for the duration of our improvement process.

For instance, over 60 percent of the ‘old hands’ liked what they got with our design on page 4, while others, along with some first-time reviewers, asked for some changes. Both groups wanted us to use darker graphics and visuals, as presented in the design on page 3, and our design on page 6 reflects that. Some testers felt this small change would make it much easier to read and comprehend the message we intended to convey.

Lesson 4: Testers have different talents. Some can tell you quite succinctly what does not suit them on a design, while others have ideas to fix a particular challenge. Both need to be part of your group of testers. No doubt, customers serving as testers can be an invaluable source of insight.

Step 5: One little tweak can make a huge difference
To illustrate, we redeveloped our sign-up button (see top right). While some clients felt this improved things, a clear connection to the My.ComMetrics.com tools was now missing, so we added some text (see right) and made sure that My.ComMetrics.com uses the same font-color as the headline (see below ComMetrics).

We used a bar-graph as the logo because it conveys numbers and data to most business people (think spreadsheet, as two testers put it). The words in the byline also have meaning for our clientele, especially tools and benchmarking.

Lesson 5: Listen to your customers! If more than 30 percent believe a change might help, seriously consider it. Focusing on the critical changes while sticking to the KISS (Keep it Simple, Stupid) principle helps improve usability.

More resources about A/B testing, usability and more

ComMetrics – Häagen Dazs or Magnum ice cream – methodology, artifacts and bias
Andrew Chen – Product design debt versus technical debt – why your blog becomes a Las Vegas experience.
US government – The DIY guide to web usability testing

Bottom line
We all crave conversion, but it is often more than getting a client to purchase your corporate strategy or landing a new contract for whatever product or service you sell.

A site needs to give people the information required to get their problems solved while keeping them engaged. This helps encourage them to continue through the marketing funnel (e.g., go from our blog to signing up and using our product and eventually upgrading from freemium to premium).

Take-aways
There are some crucial things to remember about designing a website or a product (e.g., household goods).

1. The in-law acid test: One of our customers told us that everybody needs to use an older (e.g., age 70) web surfer to test a design. If navigation and usability is such that they can easily find what they came for and complete the process you want them to, your design has successfully cleared a big usability hurdle.
2. Be thankful that your designer and clients threw out your original design suggestion: Yes, crowdsourcing CAN work for this clearly defined problem. If your clients are motivated to help, you can end up with something that much better. If you end up with the original design smiling back at you, something went wrong. Guaranteed.
3. Worry if it looks like Las Vegas: Show me a site with great visual appeal and I am pretty sure it was created without doing A/B tests, interviewing clients and usability testing with design review. Our users wanted a simple, relatively bland design. Accordingly, our desire to add color and visuals got thrown out early on the way to the final design. Sure, we have an international crowd of business folks, and you can relax because your music lovers are different. Or maybe NOT.

Please, leave a comment! We love to hear your thoughts: how have you improved how your webpage feels and looks. Here is a chance for anyone with first-hand knowledge (this means you!) to please share your insights.

P.S. – You can get updates on this blog in Twitter by following @ComMetrics. You can also get a free subscription by RSS or:

Tagged as: A/B testing, building a website, usability design, usability testing, user experience, web analytics, web usability