Conversion optimization

The problems in general analytics


General analytics like Google Analytics, Mixpanel, Optimizely, and Amplitude are incredibly flexible tools, providing hundreds of different perspectives to the behavior of your visitors and users.

However, what matters for most businesses is not all the myriad of metrics, but to understand how their product can raise interest, desire, and finally traction as a result:

The classic conversion funnel

This document explains why standard analytics fails to understand this classic conversion funnel and why a more task-oriented tool is needed.

Attention problem

Google Analytics is the standard tool for studying how people interact with your site. It focuses heavily on page views. However, it is mediocre at measuring how much visitors pay attention to your content.

You cannot see whether the visitors pay attention or not.

The bounce rate problem

Bounce rate is the ratio of visitors who visited one page. No matter how much content is consumed or how much time is spent on the landing page, the visit is treated as a bounce if the user doesn't go to another page.

As such, this metric useless for single-page applications and long-form landing pages. *

Sometimes a bounce is tracked as non-bounce. For example, when someone opens many tabs but only visits one. This kind of “page parking” is common for millennials’ *

You can interpret bounce rate to support any argument you want.

Jared Spool

Founder of User Interface Engineering

The time-on-page problem

Time on page is always zero for single-page visits. A visitor can engage for several minutes, even hours on a long-form landing page, and your analytics won't notice any difference: time on page is zero. *

So sometimes an inactive or hidden browser tab can track up to 30 minutes of idle time, causing the metric to go more off-sync with the reality.

The attention problem

As stated above, standard analytics cannot tell how soon your visitors leave, and you have no idea about the severity of your problem.

  1. Do visitors leave while forming the first impression? (0–3s)
  2. Do they leave while becoming aware of the problem? (0–7s)
  3. Do they leave while becoming aware of the solution? (0–15s)

You don't know the critical spots where the funnel is leaking, and you cannot fix them — because you don't know what works and what scares people away.

Interest problem

Standard analytics provides only one metric, time-on-site, for measuring how much interest is built on the first visit. Sadly, this metric is broken in many ways:

  1. The value is always zero (0 seconds) for all single-page visits.
  2. The value is always zero for the last page visited.
  3. The time keeps ticking on all hidden or inactive tabs and causes the metric to go further away from reality.

Think of a typical scenario where users quickly navigate to your single-page app via front page: only a quick moment on the front page is tracked and all the time on the product is ignored.

No content consumption

Interesting sites are thoroughly crawled, so to optimize for the maximum interest, you must know how much content is consumed. The standard analytics makes no effort on measuring which pages and sections receive the most attention, or how much content was consumed in total on a single visit.

No interest

Standard analytics doesn't tell you whether your visitors are interested in your content or not because there is no such measure. Essentially you cannot tell how good your site is at engaging visitors, and whether the new launch caused more harm than good, or which marketing segments are most interested in your product.

Desire problem

Standard analytics use retention reports to study how visitors come back over time. It's currently your best bet for measuring desire.

Example retention report from MixPanel

Unfortunately, these reports fail to give a straight answer to whether you are building something people genuinely want:

  1. No interest. There is no information on how much interest was build in all the visits so far — i.e., how much the visitors consumed the content and spent time engaging in total.
  2. No return rate. Event trackers don't care whether the visitors are new- or returning, so you don't know how many truly came back after the first visit.
  3. No return intensity. Someone who returned several times is more engaged than someone who came back by accident. Retention reports lack this essential info.
  4. No recency. Someone who came back recently is more likely to come back, than someone who forgot you exist. Retention reports won't reveal when the visitors stopped visiting.
  5. No score. There is no engagement score that summarizes the total engagement over time. You cannot tell how good your site is at engaging visitors, or which marketing segments desire your product the most.

Traction problem

In standard analytics, all the analytical views come with start-, and end times. These time ranges are suitable for historical trends but fail to measure traction: how much engagement, virality, and conversions are accumulated over time.

Look for single metric within a time range

The conversion rate problem

When measuring a conversion rate, you take a group of visitors, wait for some time, and see how many of them had converted.

However, that's not how regular analytics work.

They take a set of visitors within a time range and look how many converted inside the range. And if you take another time range for comparison, it has a different set of visitors in different stages of the customer lifecycle.

Time ranges don't care whether the visitors are new, returning, or long-term loyalists — or whether they had already converted before the start date.

All these cause problems. For example, when you acquire new visitors, the conversion rates will drop, because you now have more visitors that are just building interest and desire before taking action. *

No north star metric

Standard A/B testing tools only look for one metric and ignore everything else. For example, it doesn't care whether the new version causes more people to bounce, or makes them consume less content, or have worse retention numbers.

More importantly, there is no metric for traction, which should be your ultimate goal when optimizing your site. That is more leads, more customers, more loyalty, and more virality over time.

Optimization problem

Due to above problems your conversion funnel looks like this:

Mainly your decisions are based on severely limited knowledge about how the visitors convert. Moreover, the wins are likely temporary: what works on the current time range, doesn't necessarily work in the future.

You can experience this yourself with a simple A/A test with two identical variations. The results vary for months, even with a high-traffic site. *

Many businesses would be better off if they didn't run A/B tests at all.

David Kadavy

Author of "Design for Hackers"

Learn how Volument works