RU

Ryzen 9 8940HX and RTX 5060: Running Local LLMs on a Laptop with 8 GB of VRAM

Published: 2026-11-17 · Author: AI Release · @ai_release1
Ryzen 9 8940HX and RTX 5060: Running Local LLMs on a Laptop with 8 GB of VRAM

⚡ The gist

🔍 What happened

An author going by Lexus08 took a MAIBENBEN X17F gaming laptop — AMD Ryzen 9 8940HX, NVIDIA GeForce RTX 5060 Laptop with 8 GB of GDDR7, a single 24 GB DDR5 module, a 17.3-inch display — and turned it into a local AI workstation: the article on Habr. A second 24 GB module brought the total to 48 GB, but the measurements were taken at 40 GB to keep the engine comparison fair. The laptop was kept in performance mode — via a dedicated key or the profiles app — otherwise the system falls back to the integrated GPU.

The mechanics are described layer by layer. "Hot" weights and the KV cache go into VRAM, while the rest lives in system RAM and gets computed on the CPU. GPT-OSS 20B in Ollama 0.40.14 on MXFP4 got 25/25 layers on the GPU, but the MoE experts stayed in RAM — 5.0 GB of VRAM, 38%. Qwen3-Coder-30B-A3B on IQ4_NL — 49/49 layers, 5.8 GB, 29%. DeepSeek-Coder-V2-Lite on Q2_K — 16/28 layers, 5.7 GB, 37%. The heaviest setup, Qwen3.8-Flash-Next Coder IQ1_M on the Strata 0.2.10 engine (engine 0.1.56, CUDA 13), took 58.4 GB across two shards plus 5.2 GB for MTP: 23.4 GB of experts in RAM, 689 "hot" experts and a 20,480-token KV window in VRAM, with the second shard read from the SSD. Common settings: temperature 0.3, 65,536-token context, q8_0 KV cache (int8 in Strata), flash attention, reasoning disabled, one request at a time.

The "Hard Code" benchmark isn't a chat with a bot. The model gets a spec and hands back files in a "### FILE: path + code block" format; a script places them, builds the project, and runs hidden tests the model never sees. The tasks: a 16-voice C++20 DSP synthesizer with anti-aliasing, ADSR, a State Variable Filter, and voice stealing; a distributed Raft-based storage; an audit of someone else's multi-module project. Build errors get fed back to the model — up to three rounds for compilation and one for tests. Telemetry was captured once per second: temperature, watts, memory, tokens.

🧠 Our take

The value here isn't the hardware — it's the methodology. Usually "local AI on a laptop" gets tested with a haiku about a cat; here it's compilation, hidden tests, and a hard limit on fix rounds. Our take: this is far closer to real work than any chat benchmark, because the model gets caught on whether it builds, not on how pretty the answer sounds.

What it changes. 8 GB of VRAM is no longer a death sentence for 20–30B coder models — but only if you've got 40+ GB of fast RAM alongside it. And that's where an inconvenient truth surfaces: DDR5 bandwidth in a laptop is typically several times lower than GDDR7's, so "it fits" doesn't mean "it's fast." In our experience, MoE models with experts offloaded to RAM are tolerable; dense models are not.

Who wins. Owners of gaming laptops with a RAM upgrade: they don't need a pricey workstation just to try a local coder. Who loses — anyone hoping for "local AI on 8 GB out of the box": without the second RAM module, the scenario in the article simply doesn't come together.

⚠️ What we don't know

The available excerpt has no results: no winner across the two rankings, no tokens per second, no temperatures or wattage. One machine, one run, up to three automatic build-fix rounds — you can't build statistics on that. Then there's the question of the IQ1_M and Q2_K quants: that's extremely aggressive compression, and how much code quality it ate is only visible in the hidden tests, which we haven't seen. Treat the experiment as reproducible only after cross-checking against the tables in the article itself.

💬 Worth debating

  • A laptop with 8 GB of VRAM upgraded to 48 GB of RAM, or a single 16–24 GB card: which makes more sense for local coding?
  • Three rounds of automatic build fixes — an honest measure of the model, or a measure of the glue script's quality?
  • Is saving on IQ1_M and Q2_K quants justified if the code passes the tests anyway, or does quality degrade in ways you don't notice?

Forecast: within a quarter, expect a wave of similar "laptop-class" benchmarks of MoE models with experts offloaded to RAM — the methodology is reproducible and the hardware is mass-market.

Disclaimer: one source, the author's personal test on a single laptop; the conclusions have not been confirmed by an independent run.

Source: Habr

🤖 AI summary
  • Date: 2026-11-17
  • Source: habr.com
  • Tags: local, Ollama, квантование, laptops, AI, LLM, neural networks, local LLMs
  • Key facts:
    • A MAIBENBEN X17F laptop with a Ryzen 9 8940HX and a GeForce RTX 5060 Laptop GPU ran four coder models, including Qwen3-Coder-30B-A3B and DeepSeek-Coder-V2-Lite.
    • Only 29% to 38% of the weights fit in 8 GB of VRAM ; the engine offloads the rest to system RAM, and the tests ran on 40 GB of RAM.
    • VRAM isn't the only bottleneck: two shards of Qwen3.8-Flash-Next Coder were partially read straight off the SSD.
    • An author going by Lexus08 took a MAIBENBEN X17F gaming laptop — AMD Ryzen 9 8940HX, NVIDIA GeForce RTX 5060 Laptop with 8 GB of GDDR7, a single 24 GB DDR5 module, a 17.3-inch display — and turned it into a local AI workstation: the article on Habr .
    • A second 24 GB module brought the total to 48 GB, but the measurements were taken at 40 GB to keep the engine comparison fair.
    • The laptop was kept in performance mode — via a dedicated key or the profiles app — otherwise the system falls back to the integrated GPU.
    • The mechanics are described layer by layer.
    • "Hot" weights and the KV cache go into VRAM, while the rest lives in system RAM and gets computed on the CPU.
#локальные#Ollama#квантование#ноутбуки#AI#LLM#нейросети#локальныеLLM
← PreviousAlibaba ships Qwen4-Max: 2.4 trillion parameters and a 60% price cut per task

Source: habr.com · post in Telegram