How to Split Test Your Product Listings

When you’re trying to improve online sales, your decisions should be driven by data, not guessing. Split testing is one of the most effective ways to gather useful data. In this guide, let’s define what split testing is and dive into a few different ways for online sellers to approach it.

What split testing is

Split testing (also known as A/B testing) is essentially an experiment. In each experiment, you create two or more variations, such as different product titles, featured photos, or descriptions of the product. Then you see which variation in the set performs best.

A traditional website split test directs portions of traffic to these variations so you can compare how shoppers behaved. For example, did one set of visitors stay on the site longer or convert at a higher rate?

How to run e-commerce split tests

On your own website, you can split traffic to all variations simultaneously by writing code or by using a landing page software like Optimizely or Instapage. There are a ton of others, too.

Another method to split traffic is to run A/B tests on paid advertising channels like Facebook Ads or Google Ads. Setting up a test will send clicks to different destination URLs that you set. To limit testing parameters, make sure to use identical ad creative and set your campaign to rotate the creative evenly. You’ll get better data this way.

On a marketplace like Amazon, however, splitting traffic is trickier. You’ll need to rotate the variants over time and compare the results. For example, you might test Version A for three days, then update your listing to Version B and monitor Version B for the next three days. Of course, this method reveals a limitation: shoppers may behave differently on different days of the week. You’ll need to take external factors like this into account as you analyze your results.

You can decide on a number of days to run your split test and manually rotate your variants. Alternatively, you can automate the rotation with software specific to Amazon A/B testing, such as Splitly, Cashcowpro, or Listing Dojo. Each software has its own method of calculating results, so try to understand the factors each one weighs to determine a winner.

What e-sellers should split test

Generally, your split test will examine one of the following:

1. The product title
2. The listing’s main image
3. The product description
4. Secondary branded content

You can also experiment with keywords and pricing strategies to see what moves the needle.

In a true A/B test, you isolate one variable and one variable only (for example, the product description). You do not change anything else about the listing.

A multivariate test uses the same mechanism as an A/B test but compares more than one variable. Your product title, image, and description might all be different.

Multivariate testing is best when you’re looking to see which general direction to head in. For example, if you’re testing two drastically different marketing approaches or tones of voice for the same product, you might want to try a multivariate test.

But when you’re interested in a specific question, such as which product description works best, stick to an A/B test where that’s is the only variable.

An alternative to traditional split tests

When you run a live split test, the subjects of your experiment are your store’s shoppers. This creates a risk: if a new variant performs worse than your existing listing, you could lose sales and potentially damage your search ranking.

PickFu is an alternative way to split test that takes place entirely outside of your website or marketplace. PickFu is polling software that acts like an online focus group: you ask your market research question, choose the makeup of people you’d like to respond to it (a popular group is Amazon Prime members), and PickFu brings 50, 100, even 500 respondents to your poll.

An advantage of testing outside of a live environment is that your product doesn’t need to be for sale yet. You can test names for products before they launch, or find which product variations such as color or design will be more popular before going into production. You can also compare your variations to competitors, which you can’t do in a live test.

By far, the most popular test with online sellers is using PickFu to test product photos.

Bumblebee Linens polled 50 female respondents to see which image of cloth napkins they preferred. Where a traditional split test takes days or sometimes weeks, the results of this PickFu poll came back in less than 20 minutes. The 50 women decisively preferred a new photo over the old one by a three-to-one margin. When the seller updated the listing to feature the new photo, sales improved by 209%.

How will you split test?

It’s not enough to put your product online and expect the orders to roll in. Split testing is an effective means to understand how your creative choices affect sales and other performance indicators.

Traditional split tests will help you optimize your listings, but they can be time-consuming and risk an adverse effect on sales and ranking.

For many tests, PickFu is a faster, easier way to optimize product names, main images, and descriptions — without tracking codes to implement, ad campaigns to manage, or days to wait for results. It’s also valuable to test listings against competitors to identify areas for improvement.