16 GB of VRAM for Coding Agents: How r/LocalLLM Splits Memory Between the Model and Context

⚡ The gist of the debate

An RTX 5060 Ti owner with 16 GB of VRAM and 32 GB of system RAM asks for advice on what to run locally to get something resembling Claude Code or Codex. He admits right away that an upgrade is out of reach for the next few months and asks how to squeeze the most out of what he has. The conversation quickly comes down to arithmetic: how much VRAM the weights will eat, how much the KV cache will take for a long context, and what will be left for the system.

🗣 What participants say

A user going by Glittering_Brick6158 immediately calls the card the bottleneck: “The 16GB card is the real bottleneck for agent stuff, those context windows eat memory fast”. According to him, without quantization memory will be chronically short. The OP, Su9rob, doesn't argue — “I know that this is the bottleneck” — but clarifies that he can't expand his setup in the coming months, so the question is specifically about what to do with what he has.

The most detailed recipe comes from randomgenericbot. He runs a fork of llama.cpp (llama.cpp-adaptive-kv-streaming) for a larger context together with an IQ3_S quant of a model he calls wen3.8 from the ISTA-DASLab repository, and sets a 131k context in the pi agent (pi.dev). He lays out the memory breakdown directly: the model takes ~12 GB, the KV window ~2 GB, leaving about 2 GB for the desktop and system.

FREEZX replies briefly: start with the abliterated variant of “Swift-1.5 Qwen3.8 Flash Next” and aim for an IQ2_XS quant, i.e., compress memory more aggressively. The remark comes without measurements or an explanation of why exactly this variant is better — just a recommendation of what to try first.

ExpensiveParfait677 suggests a different path — Gyro quants from the agentionai repository (Qwen3.8-Flash-Next-Gyro-GGUF), variant S, if you don't want to offload experts to the SSD. According to him, on a 16 + 32 GB setup he ran variant M with a 192k context, kv8 and mtp. This is the most optimistic result in the thread in terms of context length, but he provides no test details — speed, quality, or otherwise.

⚖️ Where they agree, where they diverge

The common denominator: 16 GB is enough only if you budget memory as a whole — weights plus KV cache plus headroom for the system. Everyone who gave specifics cites numbers: about 12 GB for the model, 2 GB for the context window, and roughly 2 GB in reserve. No one claims that a local model will replace Claude or Codex one-to-one; the discussion is about what can be run at all.

They diverge on tactics. One line is to compress the weights harder down to IQ2_XS and not be greedy with context. The other is to take IQ3_S or a Gyro variant and stretch the context length via kv streaming: 131k for one participant, 192k with kv8 and mtp for the other. The real point of conflict is that no one provided comparable benchmarks on agentic tasks: how many steps the agent completes before an error, how fast it generates, whether the model degrades at low quants. There's no consensus on a specific model in the thread — there are two incompatible ways to save memory.

🧪 What to check yourself

• Verify the exact repository and quant names on Hugging Face: in the thread they are written in shorthand, and some of the names look non-standard.
• Measure consumption via nvidia-smi at 32k, 64k, and 131k context — you'll see the real cost of the KV cache on your specific card.
• Run the same agentic scenario on IQ2_XS and IQ3_S and compare not only speed but also the number of failed steps.
• Check whether kv streaming is enabled in your chosen llama.cpp fork and how much memory it actually gives back.
• Watch for offloading to the SSD: if the experts go to disk, the agent loop's speed will suffer more than from the quant.

🔗 Sources

r/LocalLLM — Local LLM recommendations for coding and agentic workflows (RTX 5060 Ti 16GB)

Disclaimer: this is a summary of a discussion, not facts verified by the editors. The opinions belong to the participants, and the numbers — to those who cited them.