Predicted A/B testing

Know in advance how your experiments and campaigns perform.

The ability to predict future traction is essential because you get to know the long-term impact of your A/B tests in a matter of days, even hours.

Predicted A/B testing / How a new blog entry likely generates traction

In traditional A/B testing tools, such as Optimizely, you can only get results for a single conversion metric on the bottom of the funnel, and you often need to wait for months to get results that make sense.

With Volument, however, you get results for all the key metrics on your conversion funnel, and you can run a new experiment every day (or a week, given the number of page views you have).

How predictions work

When predicting someone's future behavior, Volument looks for other visitors with similar behavior, and study how they behave later on.

For example, let's take someone with a highly engaging first visit and one return visit the next day. To predict future behavior, we collect all the other visitors with close-to-similar initial behavior and see how they returned and converted later on.

This is not much different from real human interaction: you are more likely to go on a second date if the first date went exceptionally well. And if things continue to go well, you are more likely to continue with the relationship, and more likely take some form of action later on.

Bouncers are easiest to predict

We collect samples with the following rules:

  1. Visitors with more similar history are preferred over less similar history
  2. Recent visitors are preferred over the old ones
  3. Visitors on the same marketing segment are preferred over other segments

When calculating the score, the best matches are consumed first, followed with the next best samples and so on. So the longer you use Volument, and the more visitors you have, the more accurate the predictions will come.

Predicted traction

Prediction accuracy

Volument makes predictions in sequence by first proceeding to day one, and then continues predicting to day two on top of the existing predictions, then moving on to the next days depending on the average visitor lifetime. On each iteration, we use the best available samples available.

The prediction accuracy or “confidence” depends on the amount and quality of the samples on each iteration.

  • Extensive benchmarking for choosing the best machine learning algorithm
  • Applying the selected algorithm to Volument data model
  • Calculation of the prediction accuracy
  • Option to choose between predicted and non-predicted results
  • Displaying predictions in a user-friendly way

Comments