TABLE OF CONTENTS
- Identify your A/B Testing Goals
- Construct a valid hypothesis
- Identify the traffic split between A/B test groups
- Read Next
Identify your A/B Testing Goals
Every A/B test should be conducted to solve a business problem and the potential impact of solving this problem should be quantifiable in the form of a KPI such as click-through rate, conversion rate, etc. This metric (goal)
should be used to identify whether the A/B test is successful. If you aren't measuring your goals then the result of your experiment is meaningless.
There are different types of goals:
Engagement Goals: Used to identify the Vue.ai module that works best for any given touchpoint. Identify the Vue,ai modules that you want to test and measure the click-through rate on the recommendations.
Revenue Goals: Used to identify the optimal journey that yields higher revenue. To identify the impact of Vue.ai modules on your website, run an A/B test against a no-treatment journey, and measure the uplift in the conversion rates. You can also choose Average Order Value or Revenue per Visit as your goal metric.
Construct a valid hypothesis
A prediction that you make before running a test is called a hypothesis. It is a statement that clearly states what change you want to make, why you want to do so and it's expected impact. Once a goal is identified, you can begin generating A/B testing ideas and hypotheses to create customer journey variations. A strong hypothesis is made of three components:
- Defining a problem
- Describing a proposed solution
- Setting criteria for success/failure and using metrics to measure results
Some examples of the valid hypothesis are listed below:
- Cross Product recommendations on the cart page will nudge users to buy more products and help increase the Average Order Value
- If the Add To Cart button is the most visible element on the product listing page, it will catch the user's attention and increase Cart Addition Rate
- Highlighting Similar products recommendations on the product page will help users discover more relevant products and hence convert better, thereby improving the Conversion Rate
Identify the traffic split between A/B test groups
It is advisable to allocate the traffic split equally (50% - 50%) for optimal test conditions. However, you can run an experiment with an unequal user allocation (10% - 20% - 70%). It is important to note that higher the difference among the groups, the longer the test will take to conclude.