A manual A/B Test activity compares two or more versions of your website content to see which version best improves your conversions during a pre-specified test period.
NOTE
In addition to the Manual (Default) A/B Test activity (discussed in this section), Target provides two additional types of A/B Test activities: Auto-Allocate and Auto-Target. See Types of A/B Testing activities below for more information.
A manual A/B Test activity (sometimes referred to as an A/B…N test) compares two or more versions of your Web site content to see which version best lifts your conversions, sales, or other metrics you identify. Use an A/B test to compare changes to your page against your default page design to determine which experience produces the best results.
Manual A/B tests are useful when you have a clear hypothesis of ways to improve your page performance based on success metrics or alternative content delivery.
Manual A/B tests are well suited for large changes that might involve new layouts or drastically different treatments of the elements. If your test design does not easily break down into individual page elements, you should run an A/B test before a multivariate test.
When you set up your A/B test, you can determine the percentage of visitors that see each experience. For example, you might split traffic evenly between the control and a second experience, or you might test out a new, more risky experience by showing it to only 5% of your audience.
NOTE
For detailed information about determining the optimum sample size for an A/B test, see Plan Your A/B Test.
When the number of different experiences exceeds five and spans two or more locations, it’s a good idea to consider an MVT test before running your A/B tests. The multivariate test shows which areas on the page are most likely to improve conversion. These areas are the locations that a marketer should focus on. For example, the MVT test might show that the call to action is the most important location for meeting your goals. After you determine which locations and content are most useful for helping you meet your goals, you can then run an A/B test to further refine the results. For example, to test two specific images against each other, or to compare the wording or colors of a call to action. By following an MVT test with one or more A/B tests, you can determine the best possible content for the results you desire.
Types of A/B Testing activities types
In addition to the manual A/B Test activity (discussed in this section), Target provides two additional types of A/B Testing activities: Auto-Allocate and Auto-Target.
Activity Type
Description
Manual A/B Test
Compares two or more experiences to see which best experience improves conversions throughout a pre-specified test period.
This section describes how to set up a manual A/B Test activity, but the steps for the other types of A/B Test activities are similar.
Auto-Allocate
Identifies a winner among two or more experiences, and then redirects traffic to the winner, increasing conversion as the test runs and learns.
To learn about the benefits of using an Auto-Allocate activity, see Auto-Allocate in How long should you run an A/B Test and Auto-Allocate overview.
Auto-Target
Uses advanced machine learning to personalize content and drive conversions by identifying multiple high-performing, marketer-defined experiences. The most tailored experience is then served to visitors based on their individual customer profiles and past behaviors of similar visitors.
The steps for creating the three types of A/B Test activities are similar. To create an Auto-Allocate or Auto-Target activity, start by creating an A/B Test activity, but when you get to the Targeting page, choose the desired traffic allocation method, as shown below:
Auto-allocate to best experience
Auto-target for personalized experience
Include recommendations inside A/B activities
You can include recommendations inside A/B Test, Auto-Allocate, and Auto-Target activities (and Experience Targeting (XT) activities). For more information, see Recommendations as an offer.
