AMD ROCm telemetry patch lets NVIDIA PAIR route to llama.cpp on R9700 node

A developer reports that they extended NVIDIA's Personal AI Router to route inference to a tuned llama.cpp server on an AMD ROCm node, and that they filled in the GPU telemetry the router does not ship for AMD hardware. The write-up appeared on r/LocalLLaMA.

NVIDIA Personal AI Router, or PAIR, is software for routing local inference across a group of machines on one network. According to the account, it discovers participating nodes, manages supported inference engines, and presents Ollama-compatible and OpenAI-compatible proxy endpoints to applications and agents. Its scheduling depends on seeing each node's GPU load. That visibility stops at NVIDIA hardware: AMD nodes report a blind fallback, which leaves the scheduler unable to see what they are doing.

PAIR recognizes two inference engines natively, Ollama and LM Studio. The developer worked around that limit instead of waiting for NVIDIA to widen it. A custom manifest fronts an existing llama-swap server, which exposes an OpenAI-compatible API, so the tuned llama.cpp build on the AMD box serves the cluster directly rather than being reachable only through an engine the router already understands. The result is a two-node cluster whose inference requests can land on the hand-tuned llama.cpp setup.

The telemetry piece uses amd-smi on ROCm 7.2.0 to expose AMD GPU utilization and VRAM, feeding the scheduler the way the NVIDIA path does. The developer says the readings are live and that utilization on the AMD node tracks the load placed on it. That change makes the node a legible routing target: without it, PAIR had no cost signal for the AMD side and the scheduler could not weigh it against the NVIDIA machine.

Verification was part of the account. The developer says the work passed the Go race detector and ran live on two R9700 cards, identified by the gfx1201 architecture. The full stack builds and runs on Linux with ROCm. The AMD node was PIN-paired into a two-node cluster with an NVIDIA box, and inventory is advertised in both directions, meaning each machine can see the other's capabilities. The pairing is cross-OS as well, with the Kubuntu node carrying the AMD and ROCm stack sitting alongside a Windows 11 machine carrying an NVIDIA GPU.

The build loop was local. The implementation was written by a local Qwen model, and the review and verification pass used local models too, a process the developer calls vibe-coded. They say it ran surprisingly well and that the review caught real issues. The excerpt provided stops mid-sentence at that point, so the remainder of the account, including what the review turned up and how it was resolved, is not available.

How far this goes is unclear. Nothing in the report changes what PAIR ships or what NVIDIA supports, and a single write-up on r/LocalLLaMA is the only account of the work. The reported behavior has not been independently reproduced, and NVIDIA has not responded. There is no sign the routing or telemetry work has been proposed upstream. The technique stands as a community workaround for AMD owners who want their ROCm nodes pulled into a PAIR cluster with usable load data.

The durability of the setup depends on updates. The custom manifest and the patched telemetry path sit outside NVIDIA's release cycle, so a change to PAIR's engine list, its manifest format, or its telemetry interface could break them. The developer does not address that maintenance question in the available text, which leaves how long the arrangement keeps working as the open item.