Is It Worth Chasing New Inference Engines: A Debate in r/LocalLLaMA

⚡ The gist of the debate

The author, posting under the handle Kahvana, believes the local inference ecosystem has grown to an unwieldy size: if in March 2025 llama.cpp, ollama, koboldcpp, vllm, exl3+tabbyapi and litert were enough, now you also have to keep Strata, Basalt, Maya, Colibri, Ninfer, and Infernix in your head. The question to the subreddit is simple: what do you use, and how do you manage to keep up with updates. The debate quickly turned to a different question — whether you need to "keep up" at all.

🗣 What participants say

mrazster answers extremely briefly: he doesn't follow and doesn't intend to — "I just use llama.cpp and don't give a shit about the rest." His argument is that there's already enough to handle: more than enough models and harnesses to keep track of.

Perfect-Campaign9551 objects to the very framing: "Why does everyone feel the need to 'keep up with it all'?" In his view, keeping up means reading the subreddit's news, not installing every engine. He learned about Strata from his feed, downloaded it about two weeks later, and now runs Strata with Qwen Next Fl.

Kooky-Mode3047 gives the factual picture: all the buzz is around Strata and MoE models like Deepseek and QFN (3.8 and the upcoming 4). Basalt, Infernix, and Strata-nvfp4, according to him, are specialized forks of Strata; Basalt is the fastest so far without losing quality, though Infernix, Strata-nvfp4, and base Strata lag only slightly.

OvertaxedOne recalculates the author's config — two RTX 5060 Ti and 96 GB of DDR5 — and advises trying Strata anyway: it supports the same Q4XL for QFN, and generation speed, by his estimate, will increase "several times over." He also formulates the thesis the thread arrived at: the "model chase" is over for many, and the choice ahead is between QFN and 27B depending on your hardware.

OvertaxedOne's second comment is about GPUs: he's on Ampere himself, and Strata, he says, is more tolerant of supported GPUs; he's heard only good things about Blackwell engines but can't personally confirm. Then a joke about a Pro6000 "on loan for testing."

⚖️ Where they agree, where they disagree

The common denominator: nobody disputes that there are too many engines now and that chasing every release is pointless. All participants, one way or another, run one or two tools. Kahvana's llama.cpp v0.8.0 delivers 15 t/s on Qwen3.8-Flash-Next Q4_K_XL (unsloth) with dual RTX 5060 Ti 16GB, PCIe 5.0 x8x8, tensor parallel, no MTP, and he's fine with that.

The conflict lies elsewhere. Kahvana measures the lag as lost speed and suspects new engines have "a catch, usually bad quality." OvertaxedOne and Kooky-Mode3047 believe that the Strata + QFN combo yields a multiple-fold gain with no quality loss. mrazster and Perfect-Campaign9551 reply that it's a matter of habit, not the engine: knowing it exists is enough. There's no consensus: nobody in the thread has publicly verified either "several times faster" or "worse quality."

🧪 What to check yourself

• Run the same model (Q4_K_XL / Q4XL) in llama.cpp and in Strata on your own hardware, compare tok/s and quality on the same set of prompts.
• Check the list of supported cards: OvertaxedOne specifically notes that Strata is more tolerant of Ampere, and he hasn't personally tested the Blackwell engines.
• Reproduce the author's config (dual 5060 Ti, tensor parallel, no MTP) and see how much enabling MTP and changing the layer layout helps.
• Check how the forks (Basalt, Infernix, Strata-nvfp4) differ from base Strata before installing — these are claimed forks, not independent projects.
• Measure not only generation but also time to first token on long context: that's where "bad quality" usually shows up.

🔗 Sources

r/LocalLLaMA — Keeping up with new inference engines

Disclaimer: this is a summary of a discussion, not facts verified by the editors. The opinions belong to the participants, and the numbers to those who cited them.