DeepSeek 284B runs on 64 GB of RAM without a GPU: how llama-modchip works

⚡ The gist in 5 seconds

• The synchopate/llama-modchip repository promises to run the 284B DeepSeek model on 64 GB of RAM without a graphics card.
• On Hacker News the post gathered 402 points and 180 comments — interest in local inference is high.
• There's only one requirement: 64 GB of memory. But with that math, the model only fits in a very aggressive quantization.

🔍 What happened

A repository appeared on GitHub: synchopate/llama-modchip. The title states it outright: 284B DeepSeek on 64 GB of RAM, no GPU. The Hacker News thread racked up 402 points and 180 comments — for the niche topic of local inference, that's a notable reaction. In other words, people didn't just scroll past the link; they went to argue.

The mechanics, if you read the name literally: "modchip" is a reference to console chips, i.e., a workaround. Our guess: the project doesn't run the model "as is," but plays with where the weights live. Then comes arithmetic anyone can verify. 284B parameters at 4 bits is roughly 142 GB, at 2 bits about 71 GB, at 1.6 bits around 57 GB. Only the last variant fits in 64 GB. That means either extreme quantization below 2 bits per weight, or the weights aren't kept in memory in full but paged from disk via mmap — in which case 64 GB is a page cache, not model storage.

A second point that often gets confused. DeepSeek has a MoE architecture: only a subset of experts is active per token. That saves compute, but not memory — all the weights still have to live somewhere. So MoE alone doesn't save you 64 GB.

🧠 Our take

Our assessment: interest in projects like this didn't come from a love of records, but from a very mundane reason — the price of graphics cards with decent VRAM. When 24 GB costs as much as a used car, people start looking for ways to squeeze big models out of plain DDR5. Hence the wave of projects around MoE models and aggressive quantization.

What does it actually change? Not much, for now. We believe the main bottleneck here isn't memory but speed: on CPU, inference is limited by memory bandwidth, not core count. And if the weights are also paged from an SSD, you're counting seconds until the first token. This isn't a GPU replacement — it's a different genre: "slow, but local and free of cloud censorship."

Who wins is clear: those who value private inference on their own machine and already have 64 GB. Who loses: those who read the headline "284B on 64 GB" and decide they now have a full frontier bot in their laptop. The quality gap between 284B at 1.6 bits and 284B in BF16 is enormous, and the repository's name doesn't spell that out.

⚠️ What we don't know

The main weak spot — we've only seen the title and description, not a README with measurements. It's unclear what exactly the "284B" is: a base model, a distill, a quantization, or a mixture of experts with pruned layers. There are no tokens per second, no exact weights filename, no RAM usage under load, no benchmark validation. The numbers above are our arithmetic from the model size, not a measurement. For now, 402 points on HN mean interest, not reproducibility.

Prediction: within a month it will become clear whether a reproducible measurement appears in the repository — tokens per second, the exact weights file, and peak RAM. If the numbers add up, expect a wave of forks targeting other MoE models.

Disclaimer: not officially confirmed, no independent measurements — all specs are taken from the project description; we did not reproduce them.

Source: GitHub / synchopate