Volument engineering • Large data study
Navigator.sendBeacon() is broken
sendBeacon is a browser method to transfer data to the backend prior to unloading of a web page. However, after studying over 5 million pageviews we can conclude that the Beacon API is broken and should not be used in production.
Created: Dec. 17, 2019
Edited: Dec. 18, 2019 — Conclusions added
Edited: Dec. 19, 2019 — iOS removed from the list
5,130,741 page views sampled • 3,217,740 successful unloads • 628 ignored browsers (less than 100 samples)
Regardless of browser version or operating system.
Regardless of the browser brand.
Browsers with the biggest amount of failures
Browsers with least amount of failures
How we got here
One major difference between Volument and others is how the page view data is collected. While others send data when the page is loaded, Volument sends data after knowing how the page and it's viewports were consumed.
Obviously, we opted for Navigator.sendBeacon() to send this page consumption info to our servers. It is absolutely a perfect fit for our use case since it is designed to deliver data to servers before the unloading of a page.
It solves all of the problems with submission of analytics data: the data is sent reliably, it's sent asynchronously, and it doesn't impact the loading of the next page.
Like most frontend developers, we use the latest browsers while developing a service, so we didn't notice anything unusual while developing the product. About 3% of the requests fail on the latest Chrome on Mac.
We were quite confident that our data tracking was properly setup. According to caniuse.com the beacon API is supported by 93.44% of the browsers, and we have a fallback data collection mechanism in place for browsers that don't support the API.
However, after using sendBeacon for a few days our beta customers’ websites, we noticed big data inconsistency between Google Analytics and Volument. Volument pageview counters were 30-40% smaller on average.
To find out the problem, one of the first things we did was to start tracking the performance of sendBeacon.
We currently have 38 beta customers sending data to our servers. Together they produce enough sample data to measure the performance of any browser-based technology in a fair amount of time. With the sendBeacon we did the following:
- Increment a counter when a page is loaded
- Increment another counter when a page is unloaded
- Measure failure from the difference of these two counters
For the page loads we used the standard POST method in the
XMLHttpRequest object and for the unloads we did the following:
We tried both
unload events with similar results: some of the calls succeed and some fail. The problem would be easy to debug if the call would always fail, but that is not the case, unfortunately. The problem is on the built-in browser method, which cannot be trusted. No browser vendor can guarantee reliable data delivery.
Our sample data is collected upon the
beforeunload event because it is fired before the unload event and is thus more likely to succeed. We used bowser for browser detection.
- On this sample set, about 30% of browsers that claim to support the beacon API failed to deliver the data to our servers when the page was closed, which is the whole purpose of the sendBeacon call.
- Linux fails 94.1% of the time
- It's extremely hard to feature-detect beacon support. The call might work on the page, but does it work when the page is closed? Seems you must look for supporting browsers from the above list and accept a 1-3% failure rate.
- The potential fallback mechanisms (XHR, Image) are likely even more unreliable when the page is closed since their delivery is not guaranteed.
- It's not you — it's the browser vendors to blame: their built-in browser implementations are broken and cannot be trusted.
Thanks to developer community feedback, we removed iOS from the list. This browser has known issues delivering on the promises of beacon API. The sendBeacon event itself is working, but the unload/beforeunload events are not fired. We should have used the
visibilitychange events instead.
Unfortunately we have no data how reliable iOS is when the data is sent on these mobile-specific events.
Unload problems are annoying to debug. Maybe the issue was due to cross-domain communication? Good guess, but why some of the calls work and some not?
In the end, we started using the (good old)
Image object for data transfer while the visitor is interacting with the page. It is the most reliable system for cross-domain communication when you don't need return values from the server.
We stopped using the unload events,
XMLHttpRequest and we stopped worrying about the missing events.
Of course, we would absolutely love to use the beacon API to send all data just once with zero performance cost. For the time being, we are choosing the old, boring, and reliable mechanism.