I’m doing some more A/B testing at my current job — I haven’t really done that much in previous jobs, although a little bit here and there — and so I’ve been looking around at videos and articles related to the idea just to see how successful it can be, how you should optimize it, etc, etc. Basically, I’m curious and trying to learn more. I think that’s a good thing.
Optimizely is somewhat of a leader in this space, and I found a pretty interesting post on their blog. They were working with InSound, which appears to be a vinyl store online, and they were working with them near the holiday season — which is obviously big money for any retailer (or, rather, can be big money). With their online sales funnel — the real term is probably “checkout flow” — they tested four combinations of wording to move people to the next stage of buying something:
- “Continue” (this was the original)
- “Submit”
- “Almost Done”
- “Review Order”
Here’s how the different variations looked:
The people at Optimizely predicted “Almost Done” would lead to the highest conversions, but as you can see above, they were wrong. Here’s the potential rationale:
But even digital optimization strategists cannot predict the habits of thousands of website visitors. Not only did “Review Order” increase conversion on the page by 8%, it beat all other variations by well over 30%. With the “Review Order” button, this page now has a 54% conversion rate – more than half of visitors move on to the next page where they actually review the order and check out. Annualizing the resulting increase in conversion rate produced a nice revenue lift and because this revenue didn’t come from additional sales & marketing spend, it’s very margin rich.
Key words in that pull-quote?
“Because this revenue didn’t come from additional sales and marketing spend, it’s very margin-rich.”
People straight-up love the margin, especially in business speak. If you can help the margin, you can please the masses (or at least those above you).
And think about this — they helped the margin and increased revenue by basically changing “Continue” to “Review Order,” which they learned from running a simple test on their page. Pretty cool, right?
Seems more useful than going out and finding users to live-test, which often costs a bunch of money.
The broader point I think this speaks to: you will sit in a lot of meetings in your life where people (managers) will attempt to say what customers/consumers like based off what they (the manager, or person speaking) do — but that’s inherently flawed. If you work for an organization, it’s almost impossible to weigh in correctly on the infrastructure of that organization’s website, especially if it’s e-commerce. You’re too close to how the sausage gets made to really understand how the people outside shop for sausage.
It’s a little bit along the lines of “customers know a product better than the managers who run the product.”
And also, lest you think these small things like word changes aren’t important — in fact, they actually are. Consider this, too: if Amazon loads even one second slower, they can lose $1.6 billion that year.
You should always keep the big picture in mind, but … the tiny word changes and site weight and all that do definitely matter.
