Volument engineering • Large data study

Navigator.sendBeacon() is broken

sendBeacon is a browser method to transfer data to the backend prior to unloading of a web page. However, after studying over 5 million pageviews we can conclude that the Beacon API is broken and should not be used in production.

Created: Dec. 17, 2019

Edited: Dec. 18, 2019 — Conclusions added

Edited: Dec. 19, 2019 — iOS removed from the list

The data

5,130,741 page views sampled • 3,217,740 successful unloads • 628 ignored browsers (less than 100 samples)

Browser brands

Regardless of browser version or operating system.

Operating systems

Regardless of the browser brand.

Broken browsers

Browsers with the biggest amount of failures

Most reliable

Browsers with least amount of failures

How we got here

Volument is an analytics service with a completely new kind of data model aiming to solve some of the known problems in general analytics.

One major difference between Volument and others is how the page view data is collected. While others send data when the page is loaded, Volument sends data after knowing how the page and it's viewports were consumed.

Obviously, we opted for Navigator.sendBeacon() to send this page consumption info to our servers. It is absolutely a perfect fit for our use case since it is designed to deliver data to servers before the unloading of a page.

It solves all of the problems with submission of analytics data: the data is sent reliably, it's sent asynchronously, and it doesn't impact the loading of the next page.

 MDN Web Docs

Like most frontend developers, we use the latest browsers while developing a service, so we didn't notice anything unusual while developing the product. About 3% of the requests fail on the latest Chrome on Mac.

We were quite confident that our data tracking was properly setup. According to caniuse.com the beacon API is supported by 93.44% of the browsers, and we have a fallback data collection mechanism in place for browsers that don't support the API.

However, after using sendBeacon for a few days our beta customers’ websites, we noticed big data inconsistency between Google Analytics and Volument. Volument pageview counters were 30-40% smaller on average.

To find out the problem, one of the first things we did was to start tracking the performance of sendBeacon.

Measurement setup

We currently have 38 beta customers sending data to our servers. Together they produce enough sample data to measure the performance of any browser-based technology in a fair amount of time. With the sendBeacon we did the following:

  1. Increment a counter when a page is loaded
  2. Increment another counter when a page is unloaded
  3. Measure failure from the difference of these two counters

For the page loads we used the standard POST method in the XMLHttpRequest object and for the unloads we did the following:

window.addEventListener('beforeunload', function {
  navigator.sendBeacon(api_url,
    new Blob([JSON.stringify(data)], { type: 'text/plain' })
  )
})

We tried both beforeunload and unload events with similar results: some of the calls succeed and some fail. The problem would be easy to debug if the call would always fail, but that is not the case, unfortunately. The problem is on the built-in browser method, which cannot be trusted. No browser vendor can guarantee reliable data delivery.

Our sample data is collected upon the beforeunload event because it is fired before the unload event and is thus more likely to succeed. We used bowser for browser detection.

Conclusions

  1. On this sample set, about 30% of browsers that claim to support the beacon API failed to deliver the data to our servers when the page was closed, which is the whole purpose of the sendBeacon call.
  2. Linux fails 94.1% of the time
  3. It's extremely hard to feature-detect beacon support. The call might work on the page, but does it work when the page is closed? Seems you must look for supporting browsers from the above list and accept a 1-3% failure rate.
  4. The potential fallback mechanisms (XHR, Image) are likely even more unreliable when the page is closed since their delivery is not guaranteed.
  5. It's not you — it's the browser vendors to blame: their built-in browser implementations are broken and cannot be trusted.

iOS

Thanks to developer community feedback, we removed iOS from the list. This browser has known issues delivering on the promises of beacon API. The sendBeacon event itself is working, but the unload/beforeunload events are not fired. We should have used the pagehide and/or visibilitychange events instead.

Unfortunately we have no data how reliable iOS is when the data is sent on these mobile-specific events.

Our workaround

Unload problems are annoying to debug. Maybe the issue was due to cross-domain communication? Good guess, but why some of the calls work and some not?

In the end, we started using the (good old) Image object for data transfer while the visitor is interacting with the page. It is the most reliable system for cross-domain communication when you don't need return values from the server.

We stopped using the unload events, sendBeacon and XMLHttpRequest and we stopped worrying about the missing events.

Of course, we would absolutely love to use the beacon API to send all data just once with zero performance cost. For the time being, we are choosing the old, boring, and reliable mechanism.

Comments

About Volument

Volument is analytics and A/B testing “for the 21st century”. It focuses solely on conversion optimization (useless for anything else).

Learn the problem we aim to solve.

Learn how Volument works

Volument reveals your best market segments and the way they gradually become engaged with your product.

View all blog entries

{"style":["/blog/pagestyle/beacon","/learn/syntax"],"title":"Large data study: Navigator.sendBeacon() is broken","data":"/blog/data/beacon-data.json","og_square":"/blog/img/beacon-og-square.png","og_image":"/blog/img/beacon-og.png","greeting":"Hungry for beacon?","date":"2019-12-17","desc":"`sendBeacon` is a browser method to transfer data to the backend prior to unloading of a web page. However, after studying over 5 million pageviews we can conclude that the Beacon API is broken and should not be used in production.","key":"sendbeacon-is-broken","uri":"/blog/sendbeacon-is-broken","created":"2019-12-15T13:27:32.569Z","modified":"2020-01-09T05:45:47.105Z","createdISO":"2019-12-15","modifiedISO":"2020-01-09"}