Compression Dictionary Transport

原始链接: https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Compression_dictionary_transport

To enable dictionary-based compression, a server providing a dictionary resource includes a `Use-As-Dictionary` header (e.g., `match="/js/app.*.js"`) specifying which resources can use it. Subsequent requests for matching resources include an `Accept-Encoding` header with `dcb` or `dcz` (delta compression) and an `Available-Dictionary` header containing a SHA-256 hash of the dictionary. An optional `id` (e.g., `id="dictionary-12345"`) in the `Use-As-Dictionary` header allows the client to provide a `Dictionary-ID` in future requests. The server must still verify the hash from `Available-Dictionary`, using the `Dictionary-ID` only as a hint. The server responds with a dictionary-compressed resource and sets the `Content-Encoding` header accordingly. To prevent caching issues, the response *must* include `Vary: accept-encoding, available-dictionary`.

This Hacker News thread discusses "Compression Dictionary Transport," a proposal to improve data compression by using pre-defined dictionaries. The core idea is that browsers and servers share a common dictionary, reducing the size of transmitted data by referencing dictionary entries instead of transmitting them repeatedly. While potentially beneficial, concerns were raised about server load due to managing multiple dictionaries and versions. Some argue that existing compression algorithms like gzip and Brotli are sufficient, while others point to examples where dictionaries offer significant gains, particularly for frequently updated JavaScript bundles or APIs with long-lived connections. The thread highlights the trade-offs involved: dictionaries add complexity, require initial download, and might not always outperform existing methods if the dictionary isn't tailored to the content. However, specific use cases like incremental updates or scenarios with limited bandwidth could benefit. Cloudflare's potential role in automatically generating and serving dictionaries was also discussed. Some worry about the potential for misuse of this technology for tracking.

原文

To use a resource as a dictionary, the server should include the Use-As-Dictionary header in the response that provides the resource:

Use-As-Dictionary: match="/js/app.*.js"

The value of this header indicates the resources that can use this resource as a dictionary: in this case, that includes any resources whose URLs match the given pattern.

When a resource is later requested that matches the given pattern (for example, app.v2.js), the request will include a SHA-256 hash of the available dictionary in the Available-Dictionary header, along with dcb and/or dcz values in the Accept-Encoding header (for delta compression using Brotli or ZStandard as appropriate):

Accept-Encoding: gzip, br, zstd, dcb, dcz
Available-Dictionary: :pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fFG4=:

The server can then respond with an appropriately-encoded response with the chosen content encoding given in the Content-Encoding header:

If the response is cacheable, it must include a Vary header to prevent caches serving dictionary-compressed resources to clients that don't support them or serving the response compressed with the wrong dictionary:

Vary: accept-encoding, available-dictionary

An optional id can also be provided in the Use-As-Dictionary header, to allow the server to more easily find the dictionary file if they do not store the diction by the hash:

Use-As-Dictionary: match="/js/app.*.js", id="dictionary-12345"

If this is provided, the value will be sent in future requests in the Dictionary-ID header:

Accept-Encoding: gzip, br, zstd, dcb, dcz
Available-Dictionary: :pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fFG4=:
Dictionary-ID: "dictionary-12345"

The server must still check the hash from the Available-Dictionary header — the Dictionary-ID is additional information for the server to identify the dictionary but does not replace the need for the Available-Dictionary header.