Solo dev runs 27B 1-bit model at 30 tok/s in browser on 6 GB laptop GPU

A solo developer has pushed a 27-billion-parameter model to 30 tokens per second inside a browser tab running on a 6 GB RTX 3060 Laptop GPU. The engine, mentria.ai, is built from scratch in WebGPU/WGSL and runs entirely client-side: no install, no server, and nothing leaves the machine, according to the developer's post on r/LocalLLaMA.

The model is Bonsai-27B, a natively 1-bit model trained and released by Prism ML. The developer repacked it for the engine, wrote the kernels that run it, and published the repack on Hugging Face at huggingface.co/mentriaai/Bonsai-27B-mentria, with full evaluation deltas against FP16 Qwen3.6-27B listed on the model card.

The compression is what makes the footprint possible. Every weight is a single sign bit with one scale per 128 weights, roughly 1.14 bits per parameter, so the full 27 billion parameters occupy 3.8 GB of GPU memory. The matmul weights account for 3.6 GB of that, with embeddings bringing the file to 3.8 GB.

The speed milestone came quickly. Two days before the post, the same model decoded at 15 tokens per second on the same laptop. The jump to 25 to 30 tokens per second came from a kernel originally written for phones, the developer said.

Decode is memory-bound, and the arithmetic explains the challenge. Generating a single word of the 27B model requires 804 GPU dispatches, 401 of which are the 1-bit matrix-by-vector kernel that streams the model's 3.6 GB of matmul weights once per word. The winning kernel exploits the fact that four 1-bit weights have only 16 possible combinations, a detail the developer cited as the key to the speedup.

The post is a single developer's report and has not been independently verified. The developer said they checked the model's quality themselves, and the full eval deltas are in the Hugging Face card, but no third-party benchmarks were cited. The engine remains a solo project, and the post does not state when or whether mentria.ai will open its kernels or offer a public demo beyond the described Chrome test on the RTX 3060 Laptop.