DeepSeek-V4-Flash-Vision-Exp runs on 10-12 RTX 3090s at 60-120 tok/s

A developer has published a fully documented setup for running DeepSeek-V4-Flash-Vision-Exp, a 285B-parameter mixture-of-experts model, on consumer Ampere hardware: 10 to 12 RTX 3090s. The build, posted to r/LocalLLaMA by user ciprianveg, reports 60+ tokens per second decode on 10 GPUs and 120+ tokens per second on 12, with vision, speculative decoding, and tool calling all working simultaneously.

The model is the full deepseek-ai/DeepSeek-V4-Flash-Vision-Exp checkpoint, not a distilled or quantized variant. It packs 285B parameters in a MoE layout with FP4 experts and FP8 attention, totaling about 157 GB of weights. The rig runs a vLLM build compiled for SM86, the compute capability of the RTX 3090, which is the key to fitting the workload on Ampere rather than newer Blackwell or Hopper cards.

The performance figures depend on the tensor-parallel and pipeline-parallel split. On 10 GPUs at TP2xPP5, the author measured 60+ tokens per second decode with DSpark speculative decoding at k=3, all under a 240 W power cap. Moving to 12 GPUs at TP4xPP3 roughly doubles throughput to 120+ tokens per second. Long-context prefill reaches about 3,500 tokens per second. Context length tops out at 1M tokens with no offload, and 4M when weights spill to system RAM.

The author says the setup is reproducible and ships a pre-built container image at ghcr.io/ciprianveg/3090-vllm tagged dsv4-flash-vision-sm86, plus a GitHub repository with build guides, start scripts, and runtime patches. The patches address a string of issues that otherwise break the stack: a DSpark propose-gate fix covering both speculative decoding and vision, a scheduler fix for mm x spec row-crossing, grammar-bitmask validation, a vision ViT out-of-memory fix, and FlashInfer workspace-lane keying.

The post is a single-author report with no independent verification, and the underlying Reddit thread and repository could not be fetched for this write-up, so the figures rest on the author's own benchmarks. The model itself is an experimental "exp" release, which typically signals a checkpoint that may change or be superseded. Still, the claim that a 285B MoE with vision and 1M-token context runs at usable speed on last-generation consumer GPUs marks a notable step for local deployment of frontier-scale models.

What remains unverified is how the setup behaves outside the author's specific configuration, including real-world stability across long sessions and whether the 240 W power cap holds under sustained load. The author has not indicated whether the patches will be upstreamed to vLLM or folded into future DeepSeek releases.