Qwen3-8B ternarised post-training retains 78.5% of FP16 accuracy, packs to 8.24 GiB

A three-author team has published an end-to-end characterisation of converting Qwen3-8B to a ternary weight format after training, reporting that the larger model absorbs aggressive discretisation better than its 4B sibling. Anirudh Malik, M Sparsh Mehra and Poojith Devan posted the work to arXiv on 8 September 2026 as arXiv:2609.09240, describing a scale-up of a pipeline previously applied to Qwen3-4B.

The conversion stacks three existing techniques: KOTMS rotation, E2M-ATQ adaptive ternarisation, and GPTQ-style error compensation, run in a weight-only A16 configuration. The authors state plainly that they do not claim these algorithms as new. Their contribution is the scale-up characterisation itself, which they frame around an external reproduction gate, matched 4B/8B capability analysis, cross-corpus perplexity, effective-bit accounting, lossless lattice-aware packing, and direct packed execution.

The headline numbers cover both quality and footprint. Across three corpora the 8B model reaches a perplexity ratio of 1.361x against the FP16 baseline, broken down as 1.318x on WikiText-2, 1.393x on C4, and 1.371x on PTB. On eight zero-shot tasks at n = 500, mean accuracy lands at 64.6% versus 72.4% for FP16. That is 78.5% chance-corrected retention, or a 7.8-point absolute cost.

The matched 4B run is the comparison that gives the result its shape. It retains 69.6% under the same pipeline, an 8.9-point gap in favour of the 8B model. The authors read this as evidence that model size improves robustness to aggressive post-training discretisation, which is the central claim of the paper.

On the deployment side, the packed checkpoint is 8.24 GiB and preserves the recorded perplexity to measurement precision. Direct packed execution reaches 15.52 tokens per second while resident in 7.35 GiB. The authors are explicit that a preliminary packed GEMV kernel remains slower than FP16 cuBLAS, so the packing work solves serialisation for the measured artefact without yet delivering a speed win at the kernel level.

The paper opens by arguing that a nominal "1.58-bit" label does not specify the deployed representation or its execution cost, which is why the authors devote sections to effective-bit accounting and lattice-aware packing rather than reporting a single bit-width figure. The 8.24 GiB checkpoint and 7.35 GiB execution footprint are the concrete answers to that accounting question for this artefact.

The authors list their own open items: broader seeds, calibration distributions, and kernel optimisation. Those caveats matter because the capability numbers come from a single conversion run at n = 500 per task, and the perplexity ratios are reported for three English-language corpora. The paper does not claim the retention figures generalise beyond the measured configuration.

What the work establishes is a validated scale-up baseline rather than a new method. The reproduction gate is presented as external, and the matched 4B/8B pairing lets a reader separate the effect of scale from the effect of the conversion recipe. Whether the 8.9-point 8B advantage holds across seeds and calibration sets is the question the authors leave open, along with any kernel work that would turn packed execution into a throughput gain over FP16 cuBLAS.