Why Brotli Adoption is Low Among Websites

Why Brotli

Brotli is able to consistently achieve better compression than Gzip. Studies conducted on real-world web assets show a 39% better HTML compression, 26% better JavaScript compression and 17% better CSS compression over Gzip at the 90th percentile. Brotli is also great at compressing newer data types like WebAssembly.

Support status

Brotli is mostly supported only on HTTPS websites (for good reason) by Firefox and Chrome. Other Chromium based browsers like Opera, Brave and Vivaldi support it too. That is quite a lot of people who can benefit from Brotli right now!

So if a considerable number of people are on Brotli-enabled browsers and it is obviously better at compression than Gzip, you would expect a lot of websites to be using it. Sadly, you would be wrong. A quick search on BigQuery shows that only 357 domains are currently serving Brotli.

So why aren’t more websites using Brotli already if it is so awesome?

Huge websites like Facebook and Google are using it in production. The main reason for the slow adoption rate among other websites is because it is not supported by a lot of the major CDN providers. CDN providers like CloudFront, Cloudflare and Max CDN normalise the Accept-Encoding header to just gzip before making the request to your origin server. Therefore you are unable to detect if the client actually supports Brotli and send a different asset to them.

Out of these 357 domains, these are the number of domains serving Brotli via a CDN. The only commodity CDN supporting Brotli seems to be CDN77 and there are some hacks to support Brotli in Fastly too.

If you are like me, you cannot wait for other CDN vendors to enable Brotli support. This is possible using some Service Worker awesomeness.

I wrote a small Service worker script to get around this. Service workers are supported in all browsers that support Brotli - so you don’t need to worry about Service worker support.

The idea is to find if the client supports Brotli and attach that information as a header (or a query string) with each request so that the server can respond differently. The service worker is a great place to do such transformations to requests since it is able to intercept any outgoing request from the browser.

However, it is not trivial to find out if the client supports Brotli from within the service worker environment. Usually, this information is broadcasted within the Accept-Encoding header. This header is a protected header as defined by the Fetch Specification and is not accessible from within the service worker.

To get around this, we download a Brotli encoded file from a URL in the install event. If the browser supports Brotli, the response would be correctly decoded. If not, it would just be junk.

Based on this, we set a variable in the service worker indicating if Brotli is supported or not and attach a header (or a query string) with every request and your server can distinguish and handle these clients appropriately. (Make sure you whitelist or vary by that header in your CDN!) However, maintaining global state in service workers is not a good idea since they are terminated by the browser after events are handled. To persist information, we store the information in IndexedDB.

This is the method used at Dexecure to enable Brotli encoding among a larger set of users. Hope to see that number of domains using Brotli go up in the next HTTP Archive crawl 😃