Analytics of our dreams

Let’s start with the core use-cases of analytics:

1. Traffic statistics: how much, which page, where from
2. Engagement how the visitors engage with the content
3. Product analysis how the registered users behave in the product

Product analysis is out of our scope and we are happy to hand that problem to other great products like Heap, Mixpanel, or Amplitude. The first two tasks, however, are the key use-cases of Volument. This article shows you how these functions can be drastically improved.

Automation

We’ve seen that a typical way to use analytics is “fire and forget”: people just set up tracking and move on to other things. People tend to have other things on our mind than website analytics. They only want the data to be available when needed. With all honesty, we behave the same way. Regardless of how polished the user interface is, or what data is there, or how beautifully the information is presented.

However, things get a bit more interesting when vital information is automatically pushed in front of your eyes. Especially when it’s something urgent that directly impacts conversions.

We’re building an intelligence layer on top of the core traffic and traction data to detect anomalies and to send notifications to Slack, email, and to any configured server.

With automation in place, the analytics software does all the analysis work for you. When Volument is silent: you can be sure that things are running smoothly. That is: no news is good news. This is definitely not a new idea, but you don’t see great analytics solutions taking out the full potential.

The above alert is just a small example of what’s possible. The true power comes when the intelligence layer is coupled with marketing campaigns and A/B testing. Let’s look closer.

Automated marketing campaigns

What is a marketing campaign? Is it a report for some landing page? Or is it analysis for a specific utm_campaing tag? Yes: those two are the most common approaches.

However, a campaign can be much more:

This is a fully automated system that detects spikes in traffic and compiles all the details on what happens while the traffic is peaking.

Let’s look at the benefits:

1. It’s all automatic: you’ll get a notification when the spike is detected when the initial performance results or “leading indicators” are available, and when the campaign is ended and the final traction numbers are available.
2. Know everything about the campaign: the referral domains, landing pages, the continents, and the countries. Most importantly, you get all the traction data: the details on awareness, interest, desire, and actions — as well as the profits, and losses.
3. Clear campaign history: see all your traffic spikes annotated and documented without extra effort. There’s no endless amount of UTM rows to hide the signal from the noise.
4. Compare all your campaigns against each other regardless of when they occurred.
5. No lost data when the UTM tags are stripped out from the URL when shared in social media. You also get more viral spread when your links are free from UTM clutter and other tracking parameters. (UTM tags are an absolute no-go in Hacker News for example).
6. Non-campaigns. Sometimes your traffic can spike like magic, without you doing a thing. You are ranked better in Google search, someone major points to your site, or an affiliate link is created. Volument auto-detects the reason for your spikes.

Automated campaigns are implemented later this year.

Automated A/B testing

Automation is a perfect fit for A/B testing and this is where things really start to get more interesting. But before jumping into the solution, let’s focus on the problems in traditional A/B testing:

• First: it takes ages to get statistically significant results. Even with decent traffic (100K monthly sessions) you literally need months to wait before you know which button is better.
• You are not supposed to make any changes on the website while the test is running or the results are distorted. This is, of course, out of the question for any serious business that wants to develop and grow.
• You can only measure the impact on one metric, typically at the very end of the funnel. Everything else is ignored: how the change impacted first impression, and the desire to move onwards the funnel. Even if you have the patience to wait for the final result, it may not mean a thing.
• You can only test minor changes, like button colors and alternate headlines. You’ll focus on irrelevant stuff rather than fixing real problems. Just ask yourself: how much a button color really matters? It doesn’t. If you don’t believe it, please wire up a test and wait half a year for the results without making other edits on your site.

The solution

Here’s a significantly more powerful system for A/B testing to be introduced in Volument:

This is an automated system that monitors changes on your site’s ability to grab visitor awareness (8 second rule) and sends you a notification if these metrics swing. For example when a bug is deployed or if some new design made a noticeable difference.

Let’s look at the benefits

1. It’s automatic. Receive notifications when an update is detected, when the short-term results are available, and when all traction figures reach statistical significance.
2. Develop like before without thinking of A/B testing. Use the deployment system of your choice (like Git), make updates when you want, and be sure that you get noticed if you do something bad.
3. Get results sooner. Volument uses leading indicators like the 8-second stay rate to measure the performance. These metrics are actionable, meaning that your edits have a direct impact on them. When you make a new design, you’ll know the impact in hours/days instead of months. For every bottom-funnel conversion, there’s tons of data at the top.

Moreover, there’s no need for crazy stuff like bayesian math, or pre-validation with A/A testing. Website optimization should not be rocket science.

And when the website development moves away from marketers to developers and designers — the website is in better hands. It looks better and works better. There’s less bloat and tracking scripts. And no ugly FOOC effects from the visual A/B testing tools.