Experimentation isn’t necessarily the first thing that springs to mind for sellers new to Amazon. After all, they seem to follow a tried and tested formula of seamless retail efficacy coupled with a customer-centric approach to vendor management. Any deviation from that formula could potentially send ripples throughout the entire Amazon chain.
Except those sellers would be wrong. Amazon is constantly developing new solutions designed to assist vendors in optimizing and marketing their product lines. In fact, there’s an entire slew of beta features in Amazon Advertising that vendors are rarely even aware of. The reason is simple: they’re still in beta mode.
A+ content may be a great way to dynamically showcase your product line. But the question is how do you know which products to showcase? And how do you know A+ content will even be successful?
The answer: A/B split testing.
Why A/B Split Test A+ Content On Amazon?
Because it’s a statistically proven method that can lead you to discover which content drives the most conversions and which practices drive the most traffic. You might think you have an ideal listing which accurately showcases your brand’s journey, but will your customers?
Not necessarily. In fact, unless you have a proven strategy for marketing, you’re going to be spending the vast majority of your time second guessing. Marketing demands a structured approach, one that’s as much a science as it is an art form. And any science will demand careful observation and testing.
A/B testing isn’t anything new in marketing. Some of the most effective digital campaigns have been launched as a result of running two or more test versions during a product’s pre-launch. But this would be the first time it’s been applied to enhanced content on Amazon. And you’re going to need to ensure your listing is up to date and contains all the relevant information your customers need.
Amazon just made it much easier for you to A/B test your A+ content. It’s a feature called Manage Your Experiments. And if you think the idea of A/B testing is too time consuming, you may be surprised how simple it is to access. And you may be surprised what an impact it can have on your content.
What Is The Manage Your Experiments Feature?
When Amazon first announced the Manage Your Experiments feature in July of last year, they promised that they could provide more statistically sound results by A/B testing two versions of A+ content simultaneously to “see which version resonates with Amazon customers.”
It’s not a particularly bold statement. But is it accurate? As with all things Amazon, they’re not particularly forthcoming with evidence to back their claims. But there’s notable precedence for it. Just consider the following:
A 2018 survey from independent marketing agency Invesp revealed that 60 percent of companies view A/B testing as one of the most valuable resources for conversion rate optimization.
Microsoft's Bing search engine reportedly yielded a 12 percent increase in revenue from a single ad change as a result of A/B testing.
According to Google, up to 20 percent of their A/B tests generated positive results in favor of a new idea being implemented
The 2012 Barack Obama campaign generated some $60 million in funding as a result of A/B testing, with online donation conversions of 49 percent and online sign-up conversions of 161 percent.
It’s been estimated that A/B testing can double the amount of CTR for PPC ad campaigns.
Mind you, it’s not particularly difficult nor expensive to run an A/B split test. But understanding and analyzing the results can be difficult if you’re not certain what you’re looking for. Luckily, the Manage Your Experiments feature makes it relatively simple for Amazon sellers to optimize.
How To Use Manage Your Experiments
To be eligible for Manage Your Experiments, you must be a Brand Representative and have at least one eligible ASIN with published A+ content that has sufficiently high traffic.
The Manage Your Experiments dashboard is accessible through the Content Experiment tab in both Seller Central and Vendor Central. To see which of your ASINs are eligible, simply enter the ones which generate the most traffic in the “Create a New Experiment” tab. By clicking “Select”, you’ll then be asked to decide which variable you wish to optimize. Be certain that you previously created a second A+ content page for testing.
By going back into the Manage Your Experiments dashboard, you can then:
Name the experiment and set the start date and duration. Generally speaking, a period of at least six weeks is recommended, although you can run it for up to ten.
Submit both A and B versions of your test experiment.
Run the experiment. ALthough Amazon indicates that you may not be able to see confirmation of approval for up to seven business days, most users report they are approved within 48 hours.
The results of both A and B versions can be viewed simultaneously in the Manage Your Experiments dashboard. Don’t be too eager to analyze the results early on, but periodically review them during the duration of your campaign as they can change drastically week from week. Analyzing those changes can yield some fairly interesting insights into your customers browsing habits that you may want to note for future campaigns.
At the conclusion of each test, Amazon will also interpret the results for you and give you recommendations about which variation is likely to have both a short term impact of 30 days on your sales and a long term impact of 12 months.
What Variables Should Be Tested With The Manage Your Experiments Feature?
Virtually anything and everything. But if you’re new to A+ content, you may not be certain which features are likely to have the greatest impact on your customers. With that in mind, here’s some of the better performing variables to use:
Product imagery vs. Lifestyle imagery There’s advantages and disadvantages to both product and lifestyle images. People can and will want to see your product’s selling points directly for themselves. But they’ll also want to see how they can relate to it in their daily life. Running this split test can reveal insights into your audience that you may never have considered before.
Headline messages and banners People react to language in subtle ways. But Amazon has protocols which are fairly specific about not using hard sales or misleading language in A+ content. Learning which phrases can engage your audience without being too forceful can be tricky. But it can teach you just where your customers’ motivations truly lay.
Layout and design The key to unified branding is consistency. That consistency applies to everything: your color scheme, your icons, your layout and even the fonts you use. But you’re not going to know which creates the most impact without testing. There’s an entire school of thought related to color psychology and branding which is beginning to find evidence from neuroscience. But marketers have known this for decades, and A//B split testing on A+ content is an ideal platform for your own experiments.
Keep in mind that you can only test one variable at a time, so you will have to wait until your initial campaign is finished before reviewing the results of another.
The Value Of A/B Testing
Amazon changes content policies constantly. But these changes aren’t arbitrary. They’re primarily based on customer preferences.
Amazon knows what content performs best. And while you can easily create a product listing without A/B testing, you’ll be second guessing what your customers really want based on your own preconceptions. Basic A+ content may be free for eligible vendors, but the content which truly drives traffic can be costly. The value of A/B testing doesn’t just mean refinement of your Amazon listing. It gives you actionable insight into which content creates the greatest impact.
And that insight alone is worth any amount of monthly subscription fees.
Color More Lines provides white glove, global account management of your ecommerce platforms so mission-driven companies can focus on new product development, branding and growth strategies. Find out more at Color More Lines.