When I talk to people new to the world of A/B testing, I all too often hear something like:
“I saw this example of a site that changed the color of their ‘Buy’ button and got a YY% increase, but when I implemented the same it converted worse than my original!”
The problem with this “dive right in” approach of testing minor design changes without a clearly defined problem is that if the new version doesn’t increase conversions you’ve learned practically nothing, and if it does work, you aren’t really sure why.
A better approach is to take one step back and instead do the following two things before diving into test variations:
Once you’ve 1) determined the problem and 2) articulated a hypothesis, you’re ready to brainstorm on actual split-test variations. With a clearly defined hypothesis you can easily come up with truly different variations, instead of testing yet another trivial tweak of the same old idea “change button color from green to red”.
Say your hypothesis is “our customers don’t trust the claims on our marketing site”. Then you might try out various solutions such as: placing a customer quote right below the headline, adding a video “proving” your claims, extending your return policy with 2 weeks, etc.
These are all substantial changes that are likely to make a real impact on the performance of the page. The result might be positive or negative, but in either case, you tested something substantial and came a step closer to learning something meaningful about your audience, instead of “they like ‘red’ buttons better than ‘green’ buttons”.
Join 22,000+ readers and get Baymard’s research articles by RSS feed or
Topics include user experience, web design, and e-commerce
Articles are always delivered ad-free and in their full length
1-click unsubscribe at any time
Super. Excellent. Its super important to have this defined before starting to play with button colors and different headlines.
Problems can be in area’s like:
- Ease of use
Then try testing with idea that solve that problem!
All great suggestion for factors that can be problematic.
Great post! A/B testing the REAL issues or actions you want, instead of every little thing will go really far. People don’t view it scientifically and just hear it works.
© 2021 Baymard Institute US: +1 (415) 315-9567 EU: +45 3696 9567 firstname.lastname@example.org