The problem: ASTC looks like noise to LZ
RDO fixes this by choosing, among encodings that look nearly as good, the ones that repeat bits the encoder has already emitted. The GPU sees a valid ASTC texture of exactly the same size, and LZ sees matches. It's a quality-for-package-size trade, and a single knob controls it.
How it works
First the image goes through the regular astcenc encode. With LZ-RDO enabled, each block also gets one extra trial encode using the most common configuration: 1 partition, 1 weight plane, endpoints quantized to at most 32 levels, and only the always-available block modes for 2D. Blocks that share a configuration are much more likely to share bits, so this gives the optimizer more chances to find repeats.
2. Measure where errors will be visible
Raw squared error (PSNR) is the wrong measure when you're adding error on purpose. Errors in smooth areas and slow gradients are easy to spot. Errors in noisy texture or near edges are mostly hidden.
So before encoding, the image is split into tiles (about 256×256, or 32³ for 3D) and each texel gets an activity weight:
- Low-pass filter the image (Gaussian, σ = 2.2).
- Take the squared difference from the original. That's the local high-frequency energy.
- Blur that energy a little (σ = 1.25) so it spreads.
- Map it to a weight that falls off with the square root of the energy.
3. Move-to-front histories for endpoints and weights
ASTC packs weights from the top of the block downward and the mode and endpoints from the bottom up. That makes splitting a block into "endpoint bits" and "weight bits" a simple bitmask. The optimizer keeps two move-to-front lists of recently used values:
- Endpoint history: mode, header and endpoint bits.
- Weight history: weight bits.
4. Build candidates for each block
For every block, the encoder scores:
- the original encoding;
- the restricted encoding;
- endpoint substitution: take endpoints from the history, compute the ideal weights for this block's pixels with those endpoints, and quantize. A quick check on endpoint-axis alignment skips pairs that can't work. The best few candidates are kept.
- weight splicing: combine every weight entry in the history with each surviving endpoint candidate that has the same weight bit count. If both halves came from the same earlier block, it's one long LZ match.
5. Score with a simple LZ cost model
Each candidate's cost is cost = weighted_SSD + λ · estimated_bits
The bit estimate is deliberately simple:
Unmatched bits are billed as literals.
- A match costs about 10 + log2(mtf_position + 32) bits.
- If the endpoints and weights match the same earlier block, they're billed as one match.
- The first two bytes (where the mode lives) are costed from an adaptive histogram, so frequently used modes get cheaper.
- A partial byte at the endpoint/weight boundary is billed as literal bits.
λ is scaled by texel count, so the same value behaves about the same at 4x4 and at 12x12.
6. Multiple passes, in parallel
The image is processed in independent chunks of blocks, so it spreads across threads. Each pass seeds the histories with random blocks from the chunk, and passes alternate between forward and reverse order. Because splicing produces the same bit pattern many different ways, a small hash-table cache of decoded blocks avoids decoding them again during error checks.
Using It
astcenc -cl albedo.png albedo.astc 6x6 -thorough -rdo-lambda 20
API:
astcenc_config_init(ASTCENC_PRF_LDR, 6, 6, 1, ASTCENC_PRE_THOROUGH,
ASTCENC_FLG_USE_LZ_RDO, &config);
config.lz_rdo_lambda = 20.0f;
Tuning guidance:
- λ is tuned so that a step of about 10 is a visible difference.
- Diffuse/albedo: 20–40 is usually fine.
- Normal maps: stay low, around 10 or even below, depending on your quality bar.
- -rdo-passes 2 (the -exhaustive default) squeezes out a bit more, at a real cost in encode time.
Limitations
- LDR only. Nothing prevents HDR in principle.
- In-memory size doesn't change. The gain only shows up after LZ compression on disk or over the wire.
- It costs encode time. It's meant to be paired with the higher-quality presets.
Results
Thanks
Huge thanks to Fabian for the optimizations, fixes and for getting this merged, to ARM for taking it upstream, and to Charles, Fabian and Sean for the Oodle Texture work this comes from.