Cherenkov streams Qwen3.8-Flash-Next experts from SSD on a 32GB M4 MacBook Air

A developer posting to r/LocalLLaMA as /u/alfredr has released Cherenkov, an inference engine for Apple Silicon that runs Qwen3.8-Flash-Next at 8 to 22 tokens per second at Q4 or roughly Q4 precision on a 32GB M4 MacBook Air. The author says he may currently hold the record for memory-constrained inference of that model on Apple Silicon, and puts the engine's footprint at about 21GB of allocations. The announcement carries the title "Faster than Light in Air."

Getting a model of that scale onto a laptop with 32GB of shared memory is the problem Cherenkov is built around. The engine keeps a bounded working set of experts in unified memory instead of loading the entire model, and combines predictive expert streaming with optional mixed-precision execution, according to the announcement.

The streaming is predictive. A one-layer lookahead predicts which experts the model will need next and initiates SSD reads ahead of the point where the forward pass calls for them. The reported runs use Q4 or approximately Q4 quantization, and the author describes mixed-precision execution as optional rather than required. The post does not say which parts of the stack are open source or where the code can be obtained.

The numbers are the substance of the claim and also its thinnest part. Qwen3.8-Flash-Next is reported at 8 to 22 tokens per second, a spread wide enough to cover very different operating points, with no breakdown of context length, generation length, or workload. The 21GB figure is what the author offers as evidence for the record, measured against a 32GB machine. No baseline or competing engine is named, and the post describes no measurement methodology.

The post lands in familiar territory for r/LocalLLaMA, where the constraint that decides what runs locally is memory rather than raw throughput. The engine's premise is that not every expert has to sit in unified memory at once if the ones needed next can be fetched in time, and the whole approach turns on whether the lookahead keeps storage reads ahead of demand. The title plays on Cherenkov radiation, the glow emitted when a particle travels faster than light through a medium rather than through vacuum.

The material available for assessing the claim is the announcement itself. No repository or benchmark harness accompanies it, and no third party has reproduced the result, so the record assertion rests on the author's own measurement. Whether a single-layer lookahead predicts experts accurately enough to stay ahead of the forward pass, and how the engine behaves at longer contexts, are questions the post leaves open.

Reproduction on comparable Apple Silicon will decide how much the figures are worth. Independent runs at the same quantization would test both the 21GB allocation figure and whether the 8 to 22 tokens per second range survives contact with other workloads. Until those appear, Cherenkov is one developer's report about his own machine, presented as such.