Technical details

How Volument works

Abstract

Take any marketing book or growth publication, and they show you this process of converting first-time visitors into something more valuable:

Conversion funnel

Volument is a special breed of analytics that focuses solely on this conversion funnel: how the different groups of people pass it, where they drop, and how to maximize its throughput.

Everything from data collection, data modeling, and data visualization is designed explicitly for solving this one task.

However, being such a specific analytical tool, Volument is entirely useless for anything else than conversion optimization. For example, Volument cannot be used to measure your system uptime, while general event trackers such as MixPanel, Heap, and Amplitude are perfect for that.

Volument is custom-built for one purpose, and this specificity is the main difference between generic analytics solutions and Volument.

Measurement is fabulous. Unless you're busy measuring what's easy to measure as opposed to what's important

Seth Godin

Attention

One significant difference between Volument and other analytical tools is how the raw data collected: Volument collects data when the visitor leaves a page, while the others collect data when the page is loaded.

Active time gives us a unique ability to see how much visitors pay attention to the content. For example, we can provide answers to the following when the visitors first arrive at your site:

  1. Do things look good in general? (stayed 3+ seconds)
  2. Is the problem relevant to me? (stayed 7+ seconds)
  3. Do I trust the solution? (stayed 15+ seconds)

If the landing page fails to give a good answer to the above questions, it's easy for the visitor to hit the back button and leave.

First impression

Volument measures 3s stay rate, which is the ratio of visitors who stayed longer than three seconds. If the majority of visitors drop at this stage, the problem is likely the following:

Slow load time 53% of visits are abandoned if a mobile site takes longer than 3 seconds to load. *

Layout problem Page layout is complex, and the content is hard to access or hidden behind a sales-overlay.

JavaScript error The page is broken because of an error.

Auto-play problem A non-muted video or audio starts automatically.

Issues at this stage are easiest to fix because the first impression is 94% related to design. *

Data tracking

While others use an image pixel, or similar, to track data on page load, Volument uses browser's built-in sendBeacon method to transfer data to a web server before unloading of the web page.

Unfortunately only ~95% of browsers support this method reliably upon page unload so we fall back to a sequence of XMLHttpRequest posts while the page is consumed. We start by sending an update on every second, and after five seconds, we only send data if the visitor does something with their mouse or keyboard.

We continuously track the reliability of sendBeacon across browsers and write about the results shortly.

Google Analytics doesn't know how soon people leave. Learn more…

Problem awareness

Volument measures 7s stay rate, which is the ratio of visitors who stayed longer than seven seconds. If the majority of your visitors leave at this stage, the issues are likely the following:

  1. Wrong market the problem is not relevant for the visitor
  2. Wrong problem the problem is not worth solving at all

We can fix the first one with better targeting, but if no amount of targeting works, you are likely working on a problem that nobody cares.

Solution awareness

Volument measures ‘15s stay rate`, which is the ratio of visitors who stayed longer than 15 seconds. If the majority of visitors leave at this stage, the issue is likely the following:

  1. Bad intro — people don't understand the solution
  2. Bad execution — people doubt the implementation

If 55% of your visitors pass this phase, you are already better than most sites on the Internet. These remaining visitors are your suspects. *

Interest

Activity monitoring

Another unique feature in Volument is activity monitoring. We check for every mouse movement and keyboard event to know whether the visitor is focusing on the content and not doing anything else.

This monitoring gives us the ability to measure the time engaged — the time that was actively spent with the content. The time tracking stops when the visitor is daydreaming or chatting with friends.

For example, Medium calls this “total time reading” (TTR), and for them, it's “the only metric that matters.“ *

ChartBeat made an eye-tracking study on how users focus on a web page. They realized that if the reader isn't doing anything with the mouse or keyboard within five seconds it is 95% certain to assume the user is not paying attention to the content. Volument uses this insight on all-time calculations. *

Viewport consumption

In addition to active time, Volument monitors all changes on the scroll position to measure how all the viewports on the page are consumed, which is another unique feature in Volument.

We track this data for two important reasons:

  1. We see how thoroughly the site and all the pages are crawled
  2. We know what content works, and what scares people away

We declare a viewport consumed when a visitor stays on it for a sufficient amount of time that is enough to digest the content entirely. This time depends on the size of the viewport: smaller screens require less time, and bigger screens require more.

Learning how a change affected to page consumption

Scoring

With activity monitoring and viewport consumption, we can see how much interest visitors build over a single visit. Volument does this by compiling a score from the following factors:

  1. How much time visitors actively spent with the content
  2. How many viewports they consumed during that time
  3. How many pages were visited

Here are the possible scores and their respective grades according to US school system:

  • A (70-100): exceptionally strong interest
  • B (50-70): strong interest
  • C (30-50): some interest
  • D (10-30): weak interest
  • F (0-10): no interest

Interest works as a powerful predictor for a return visit. Chartbeat found out that active time alone can predict whether the visitor is coming back. For example, visitors who engage for three minutes returned twice as often as those who engage for one minute. *

Desire

The biggest difference, however, is on how Volument models data. Instead of just studying some specific time range, Volument starts tracking visitor behavior from the first-ever interaction with the product. It then collects all the events from this same visitor and appends this data chronologically as follows:

The initial crush

We can see how the relationship started, how intensively it continued, and how the initial crush stabilized. That is: how the visitor became engaged with the product.

Rather than being an event database, Volument can be described as a relationship database. For example, we can pick any day from the timeline and measure how engaged the visitor is using the following information:

  1. Total interest — how much interest was build so far?
  2. Total amount of visits — how many times the visitor came back?
  3. Days since the last visit — when did the visitor stop coming back?

Volument uses a popular RFE model to calculate the score. The initial letters stand for recency, frequency, and engagement. *

Market demand

Volument can study how a group of visitors engage by placing all the visitors on the same line as follows:

Group of visitors studied over time

These cohorts are much different from what you have seen in other analytical tools for two outstanding reasons:

  1. Instead of cropping the visiting data with a time range, we start measuring from the very first visit. This is the only way to see how all the marketing segments start engaging with the product. Essentially the cohorts are comparable since they all measure the conversion funnel performance from the same starting point: the first-ever visit.
  2. Instead of looking only for retention, but study all the different engagement metrics to see how visitors build interest, desire, and traction.

This score indicates market demand works as a powerful predictor for future traction:

  • A (70-100): exceptionally strong market demand
  • B (50-70): strong demand
  • C (30-50): some demand
  • D (10-30): weak demand
  • F (0-10): no demand

Depending on the site's ability to retain visitors, Volument measures the total engagement for days 1, 3, 7, 14, 30, 60, and 90.

Action

When people desire a product, they start converting to a higher level in the relationship:

  1. They sign-up and become members, testers, or users.
  2. They buy stuff and become a customers and repeat customers.
  3. They share the product and become promoters spreading the word virally on your behalf. Promoters who invite multiple peers are sometimes called evangelists.

Volument tracks these conversion events and places them on the same line together with the visiting history:

Visitor engagement and conversions over time

Lifetime value

With all the engagement and conversion data in place, Volument can calculate the lifetime value from everything that makes the visitor valuable. This score is a combination of following

  1. Conversions — the number of conversions and the conversion level, i.e., paying customers are more valuable than members on the mailing list
  2. Viral co-efficient — how many peers the visitor was able to bring into the product and the value of those peers
  3. Recency — when did the visitor stop coming back?

Recency in particular (days since the last visit) is the single most important factor on visitor value because a highly retained visitor is more likely to come back and bring in more conversions: share the product more, buy more products, or upgrade to a bigger plan. *

Traction

Traction — the total amount of engagement, conversions, and virality built over time. It is referred to “a north star metric” because it's the single best metric for tracking accumulated user value for a product over a period of time. *

This value can be measured by placing the engagement and conversion histories on the same line as follows:

Total conversions over time

We calculate this metric for days 1, 3, 7, 14, 30, 60, and 90, depending on how well you can retain visitors and the value indicates the strength of your business model:

  • A (70-100): viral business model
  • B (50-70): strong business model
  • C (30-50): average business model
  • D (10-30): weak business model
  • F (0-10): failing business model

Predictions

Volument attempts to predict how visitors behave in the future. This is important for two reasons:

  1. We can compare newly acquired market segments with the more mature ones.
  2. We can see the long-term impact of our experiments early on.

The ability to reliably compare the different segments is the basis for optimizations. For example, we can see in advance how a newly acquired marketing segment behave compared to all the current visitors (the baseline), or how much traction a new product launch will bring compared to what the situation was before the launch.

How predictions works

When predicting someone's future behavior, Volument looks for other visitors with similar behavior, and study how they are behaving later on.

This behavior is not much different from real human interaction: you are more likely to go on a second date if the first date went exceptionally well. And if the dates continue to go well, you are more likely to continue with the relationship, and more likely take some form of action later on.

Predicted traction

For example, let's take someone with a highly engaging first visit and one return visit the next day. To predict future behavior, we collect all the other visitors with close-to-similar behavior and see how they returned and converted later on.

We collect samples using the following rules:

  1. Visitors with more similar history are preferred over less similar history
  2. Recent visitors are preferred over the historical ones
  3. Visitors on the same marketing segment are preferred over other segments

When calculating the score, the best matches are consumed first, followed with the next best samples and so on. So the longer you use Volument, and the more visitors you have, the more accurate the predictions will come.

Bouncers are easiest to predict

Prediction accuracy

Volument predicts the future in sequence by first proceeding to day 1, and then continue predicting to day 2 on top of the existing predictions, then moving on to days 7, 14, 30, 60 and 90. On each iteration, we use the best available samples available.

The prediction accuracy or “confidence” depends on the amount, and quality of the samples on each iteration.

Check out live demo