Large Data Study: Navigator.sendBeacon() is broken

How we got here

Volument is an analytics service with a new kind of data model aiming to solve some of the known problems in general analytics.

One major difference between Volument and others is how the page view data is collected. While others send data when the page is loaded, Volument attempts to send the data after knowing how the page and it’s viewports were consumed.

Obviously, we opted for Navigator.sendBeacon() to send this page consumption info to our servers. It is absolutely a perfect fit for our use case since it is designed to deliver data to servers before the unloading of a page.

It solves all of the problems with submission of analytics data: the data is sent reliably, it’s sent asynchronously, and it doesn’t impact the loading of the next page.

MDN Web Docs

Like most frontend developers, we use the latest browsers while developing a service, so we didn’t notice anything unusual while developing the product because, as we later found out, only 3% of the requests fail on the latest Chrome on Mac.

We were quite confident that our data tracking was properly setup. According to caniuse.com the beacon API is supported by 95% of the browsers, and we used fallback data collection mechanism in place for browsers that don’t support the API.

However, after using sendBeacon for a few days our beta customers’ websites, we noticed big data inconsistency between Google Analytics and Volument. Volument pageview counters were 30-40% smaller on average.

To find out the problem, one of the first things we did was to start tracking the performance of sendBeacon.

Measurement setup

We currently have 38 beta customers sending data to our servers. Together they produce enough sample data to measure the performance of any browser-based technology in a fair amount of time. With the sendBeacon we did the following:

1. Increment a counter when a page is loaded
2. Increment another counter when a page is unloaded
3. Measure failure from the difference of these two counters

For the page loads we used the standard POST method in the XMLHttpRequest object and for the unloads we did the following:

window.addEventListener('beforeunload', function {
  navigator.sendBeacon(api_url,
    new Blob([JSON.stringify(data)], { type: 'text/plain' })
  )
})

We tried both beforeunload and unload events with similar results: some of the calls succeed and some fail. The problem would be easy to debug if the call would always fail, but that is not the case, unfortunately. The problem is on the built-in browser method, which cannot be trusted. No browser vendor can guarantee reliable data delivery.

Our sample data is collected upon the beforeunload event because it is fired before the unload event and is thus more likely to succeed. We used bowser for browser detection.

The visibilitychange event

Due to our measurements, the official MDN documentation now recommends using “visibilitychange” event together with sendBeacon in order to fix the situation. However, we expect it to perform even worse. Try it yourself: create a test page where you send the relevant events to the opener window using postMessage after closing a background tab. The “beforeunload” and “unload” events work every time and the “visibilitychane” event is never fired.

More debate on this GitHub thread.

Conclusions

1. On this sample set, about 30% of browsers that claim to support the beacon API failed to deliver the data to our servers when the page was closed, which is the whole purpose of the sendBeacon call.
2. Linux fails 94.1% of the time
3. It’s extremely hard to feature-detect beacon support. The call might work on the page, but does it work when the page is closed? Seems you must look for supporting browsers from the above list and accept a 1-3% failure rate.
4. The potential fallback mechanisms (XHR, Image) are likely even more unreliable when the page is closed since their delivery is not guaranteed.
5. It’s not you — it’s the browser vendors to blame: their built-in browser implementations are broken and cannot be trusted.

iOS

Thanks to developer community feedback, we removed iOS from the list. This browser has known issues delivering on the promises of beacon API. The sendBeacon event itself is working, but the unload/beforeunload events are not fired. The visibilitychange event is probably the right event on mobile devices.

Our workaround

Unload problems are annoying to debug. Maybe the issue was due to cross-domain communication? Good guess, but why some of the calls work and some not?

In the end, we started using the good old Image object for data transfer while the visitor is interacting with the page. Something like this:

function sendSomething(data, done, fail) {
  var img = new Image()
  img.onload = done
  img.onerror = fail
  img.src = 'https://tracking.volument.com?' + $.param(data)
}

We call this method several times while the page is visible and the person is moving the mouse or pressing the keyboard. This way we know the visitor is still on the page and the session is alive. We started getting 100% reliable results; there are no missing calls or CORS security concerns. Image is probably the most solid way to transfer data when you don’t need any responses from the server.

Of course, we would absolutely love to use the beacon API to send all data just once with zero performance cost. For the time being, we are choosing this old, boring, and reliable mechanism.