Beacon API is broken

How we got here

Volument is a new kind of analytics service focusing on privacy, insights, and content. One major difference between Volument and others is how the page view data is collected. While others send data when the page is loaded, Volument attempts to send the data after knowing how the page and it's viewports were consumed.

Obviously, we opted for Navigator.sendBeacon() to send this page consumption info to our servers. It is absolutely a perfect fit for our use case because page engagement data is only available after the visitor has consumed the content.

It solves all of the problems with submission of analytics data: the data is sent reliably, it's sent asynchronously, and it doesn't impact the loading of the next page.

MDN Web Docs

Like most frontend developers, we use the latest browsers while developing a service, so we didn't notice anything unusual while developing the product because, as we later found out, only 3% of the requests fail on the latest Chrome on Mac.

We were quite confident that our data tracking was properly setup. According to caniuse.com the beacon API is supported by 93.5% of the browsers (May 2022), and we used fallback data collection mechanism in place for browsers that don't support the API.

However, after using sendBeacon for a few days our beta customers’ websites, we noticed big data inconsistency between Google Analytics and Volument. Volument pageview counters were 30-40% smaller on average.

To find out the problem, one of the first things we did was to start tracking the performance of sendBeacon.

Measurement setup

We currently have 38 beta customers sending data to our servers. Together they produce enough sample data to measure the performance of any browser-based technology in a fair amount of time. With the sendBeacon we did the following:

1. Increment a counter when a page is loaded
2. Increment another counter when a page is unloaded
3. Measure failure from the difference of these two counters

For the page loads we used the standard POST method in the XMLHttpRequest object and for the unloads we did the following:

window.addEventListener('beforeunload', function {
  navigator.sendBeacon(api_url,
    new Blob([JSON.stringify(data)], { type: 'text/plain' })
  )
})

We tried both beforeunload and unload events with similar results: some of the calls succeed and some fail. The problem would be easy to debug if the call would always fail, but that is not the case, unfortunately. The problem is on the built-in browser method, which cannot be trusted. No browser vendor can guarantee reliable data delivery.

Our sample data is collected upon the beforeunload event because it is fired before the unload event and is thus more likely to succeed. We used bowser for browser detection.

The `visibilitychange event

Due to our measurements, the official MDN documentation now recommends using visibilitychange event together with sendBeacon. We're going to make another test with this event after the summer and reveal our results on this blog. We expect it to give similar results, but we'll see.

Conclusions

1. On this sample set, about 30% of browsers that claim to support the beacon API failed to deliver the data to our servers when the page was closed, which is the whole purpose of the sendBeacon call.
2. Linux fails 94.1% of the time
3. It's extremely hard to feature-detect beacon support. The call might work on the page, but does it work when the page is closed? Seems you must look for supporting browsers from the above list and accept a 1-3% failure rate.
4. The potential fallback mechanisms (XHR, Image) are likely even more unreliable when the page is closed since their delivery is not guaranteed.
5. It's not you — it's the browser vendors to blame: their built-in browser implementations are broken and cannot be trusted.

iOS

Thanks to developer community feedback, we removed iOS from the list. This browser has known issues delivering on the promises of beacon API. The sendBeacon event itself is working, but the unload/beforeunload events are not fired. The visibilitychange event is probably the right event on mobile devices.

Our workaround

In the end, we started sending while the visitor is interacting with the page.

Of course, we would absolutely love to use the beacon API to send all data just once with zero performance cost. Now we need to send multiple events, which is not as cool and burdens our servers.

Comments

View all blog entries