A llama.cpp fork, a custom quant, and rare hardware: why such reports can't be reproduced

⚡ The gist of the dispute

The opening post is a parody of a genre everyone in r/LocalLLaMA recognizes: "a llama.cpp fork + a specific quant + an ultra-specific hardware config = incredible and irreproducible pp and tg" (prompt processing and token generation). The joke hit a nerve: right beneath it, a practical request quickly grew — how to run QFN on a mixed setup of an RTX 5090 and a 7900 XTX. The thread discusses what such publications lack to be believable and repeatable.

🗣 What participants say

A user going by suprjami picks up the sarcasm and lists the recipe for "that very" post: "Only if it's vibe coded and completely impenetrable to read, with the post wall of text written by LLM too". In other words: unreadable code, a wall of text, and no way to reproduce the result.

sheetis recognizes himself in this: he is "currently running a fork of a fork of Strata" with optimizations for two AMD 7900 XTX cards, even though he has four devices, and spent half a day sending "robots" to figure out the local build. The result — "runs numbers that nobody would believe", and the admission: "If I posted my numbers, I'd be that guy 😢".

Sea_C turns the conversation practical: he needs a way to run QFN simultaneously on a 5090 and a 7900 XTX via Vulkan or ROCm, and he writes outright that he is "struggling to find that obscure llama.cpp fork/config". As for the 27B, he's already got it working: "Works amazingly for 27B".

sheetis replies that with such a pairing "I'd almost certainly have to be Vulkan or bust", which means llama.cpp, which isn't the fastest path for QFN. He calls it "a long uphill battle" and adds that a "just" 5090 on Strata might surprise performance-wise — with the caveat that a lot depends on the quants.

Apprehensive_Stop314 hits the format's weakest spot with a single phrase: "Specific quant that you never mention, that is". That is the main complaint about reports that list the hardware and the numbers but omit the quant.

⚖️ Where they agree and disagree

The common denominator: a report of the form "my fork delivered X tok/s" without details is useless, and everyone in the thread admits it — some with irony, some with frustration. In effect, all four are discussing one property of such posts: irreproducibility.

They diverge in their reactions. suprjami and sheetis opt for self-irony and caution — not publishing numbers so as not to become "that" author. Sea_C wants the opposite: a concrete config and a working recipe for a mixed setup. There is no consensus in the thread on what a good report should look like: the advice about a 5090 on Strata is sheetis's hypothesis, not a verified result.

🧪 What to check yourself

• Pin down the build point: the commit hash of the fork and its parent fork. Without that, any pp/tg figures are one-off observations.
• Record the quant in full — type, who built it, file size — plus the launch parameters: -ngl, --tensor-split, batch size, context.
• Run the benchmark twice: cold start and after warm-up, pp and tg separately, on the same fixed-length prompt.
• Compare against upstream llama.cpp on the same model and quant — the difference will show the fork's contribution.
• For an NVIDIA + AMD pairing, test the Vulkan backend on both cards and separately the "5090 only" mode, as sheetis suggests.

🔗 Sources

r/LocalLLaMA — [llama.cpp fork]: QFN + [specific quant] + [ultra specific hardware config] = [incredible and irreproducible pp and tg]

Disclaimer: this is a summary of a discussion, not facts verified by the editorial team. The opinions belong to the participants, the numbers to those who cited them.