RU

Strata Squeezed a 50GB LLM Into 16GB of VRAM: 512K Context on a PC

Published: 2026-11-16 · Author: AI Release · @ai_release1
Strata Squeezed a 50GB LLM Into 16GB of VRAM: 512K Context on a PC

⚡ The gist

🔍 What happened

A blogger writing for RUVDS on Habr under the handle Tirarex put together a test rig: Ryzen 9 7945HX, 64GB of DDR5-5200 CL40, a Palit RTX 5060 Ti 16GB, and a Formula PSFX-1000 Platinum PSU. The occasion: the Strata engine, built specifically for the Qwen3.8-Flash-Next architecture and the upcoming Qwen4.

The mechanics are simple. Strata doesn't shuttle model layers back and forth between the GPU and system memory. The most-needed "experts" live in VRAM, the rest sit in RAM, and a large lookup table lives on the SSD. When the expert you need happens to be in RAM, throughput dips — but not to zero. Similar mechanisms already exist in llama.cpp; the difference is that Strata is noticeably easier to get running — and that's exactly what the author's recommendation rests on.

As a baseline, he ran Qwen3.8-27B (Q3_K_XL) in LM Studio with 8K context: 5–8 tokens/s, and after 23 minutes — an error. The context wasn't big enough to finish the Minesweeper code. Then the same prompt went to Strata running Qwen3.8-Flash-Next (IQ3_S) with 512K context: 56 seconds, 24.7K tokens, speeds from 70 down to 35 tokens/s. The game works. The billiards mode is "just for fun."

Then the author himself reveals the catch. All the demos actually ran on an RTX 2000 ADA 16GB with GDDR6 and a 128-bit bus — not on the 5060 Ti. On the 5060 Ti, he says, it should be faster, but that rig still needs to be set up.

The hardware support is all over the map: AMD from the 6000 series up to Strix Halo, Intel ARC and PRO, Nvidia from the 20-series through the 50-series, Apple Silicon, DGX Spark. On recent Nvidia and AMD cards, you install the Strata Desktop APP — a couple of clicks including the model download. On everything else, it's manual — and that's a very different evening. Full details are in the write-up on Habr.

🧠 Our take

The real bottleneck was context, not speed. 16GB of VRAM can handle a 27B model at Q3 — but only with 8K of context, and for code that's not enough: the model loses the thread and the work falls apart. Hence the 23 wasted minutes. Our read: the big shift here isn't tokens per second — it's that the promise of "512K context on a gaming PC" suddenly sounds plausible.

What does that mean for your wallet? One mid-range GPU plus 64GB of DDR5, and you're no longer paying for a subscription. But 35–70 tokens/s on a prototype and 35–70 tokens/s on a real project are two different things. We think this is fine for scripts and small utilities; for a large codebase, the question is how well the model holds long context — not how fast it reads it.

Who wins? Owners of 16GB cards and Apple Silicon — for the first time they get a clear set of instructions instead of a wall of flags. Who loses? Nobody yet: cloud APIs won't be shaken by a rig on one desk. Our hypothesis: the niche for local engines is privacy and offline use, not chasing frontier models.

⚠️ What we don't know

It's one rig, one person's measurements, and there isn't a single number from the RTX 5060 Ti in the piece. What Qwen3.8-Flash-Next actually is, and where the IQ3_S quant came from, the author doesn't explain — don't take the name at face value. There's no comparison against llama.cpp on the same model and the same context either, so "easier" refers to installation, not results. Worth testing yourself: throughput on your own hardware, and code quality after a few hundred thousand tokens of context.

💬 Worth debating

  • A rig with 16GB of VRAM and 64GB of RAM vs. a cloud subscription: which is cheaper once you count the hours spent on setup and untangling someone else's gotchas?
  • Can 512K context count as a working figure when the demo used 24.7K tokens and the author says nothing about quality over the long haul?
  • Who needs engines like this more: people with a gaming PC gathering dust, or providers who'll end up with less business?

Forecast: within the quarter, at least one independent benchmark of Strata on an RTX 5060 Ti 16GB will surface — without it, the whole "Claude at home" debate stays a story about one desk.

Disclaimer: one source, enthusiast measurements, no independent verification.

Source: Habr / RUVDS

🤖 AI summary
  • Date: 2026-11-16
  • Source: habr.com
  • Tags: local, Qwen, Strata, VRAM, inference, AI, LLM, neural networks
  • Key facts:
    • The Strata engine ran Qwen3.8-Flash-Next (IQ3_S) on 16GB of VRAM and 64GB of RAM — with 512K context.
    • A Minesweeper clone was written in 56 seconds: 24.7K tokens, speeds from 70 down to 35 tokens/s.
    • It installs in a couple of clicks via the Strata Desktop APP — but only on recent Nvidia and AMD cards.
    • A blogger writing for RUVDS on Habr under the handle Tirarex put together a test rig: Ryzen 9 7945HX, 64GB of DDR5-5200 CL40, a Palit RTX 5060 Ti 16GB, and a Formula PSFX-1000 Platinum PSU.
    • The occasion: the Strata engine, built specifically for the Qwen3.8-Flash-Next architecture and the upcoming Qwen4.
    • The mechanics are simple.
    • Strata doesn't shuttle model layers back and forth between the GPU and system memory.
    • The most-needed "experts" live in VRAM, the rest sit in RAM, and a large lookup table lives on the SSD.
#локальные#Qwen#Strata#VRAM#инференс#AI#LLM#нейросети
← PreviousF6 open-sources an MCP server for its XDR: 12 tools for the SOCNext →Telegram Disables HTML Widgets After RCE: Lockdown Mode Arrives in macOS Beta

Source: habr.com · post in Telegram