# How to Run Meta's New Open Model Locally and Compare It with DeepSeek and Qwen

> AI Release · @ai_release1 · https://ai-release.org/guides/kak-zapustit-novuyu-otkrytuyu-model-meta-lokalno-i-srav_en.html

You have a new open-weights model from Meta on your radar and you want to know whether it beats DeepSeek and Qwen for your own work — without paying per token. This guide walks you through installing a local runner, downloading the Meta model, exposing it as a local API, and running the same fixed set of prompts against all three families so the comparison is actually fair. It fits Linux, macOS and Windows 10/11 (WSL2 recommended), and assumes zero prior experience with local inference.

## Requirements and Preparation
![Illustration: Requirements and Preparation](https://ai-release.org/guides/img/kak-zapustit-novuyu-otkrytuyu-model-meta-lokalno-i-srav-en-1.jpg)


- A 64-bit machine with Linux (Ubuntu 22.04+), macOS 13+, or Windows 10/11. On Windows, install WSL2 first — the tooling below behaves like Linux there.
- A local runner. Easiest options: **Ollama** (a command-line tool that downloads and serves models with one command) or **LM Studio** (the same idea, but with a graphical window). Pick one and stick with it.
- Free disk space: 20–30 GB if you plan to test several models. Each downloaded model is one or two large files.
- Enough RAM/VRAM (memory on your graphics card). Rule of thumb: a **quantized** model — a model whose weights were compressed to fewer bits, e.g. 4-bit — needs roughly its file size in free memory, plus room for the context (the text the model can "see" at once). If it does not fit, it spills into regular RAM and gets much slower.
- `curl` (a command-line downloader) on Linux/macOS; PowerShell as Administrator on Windows.
- No API key for local runs. You only need a key if you also want to call a hosted DeepSeek or Qwen endpoint for comparison.
- One plain text file with 5–10 test prompts. This is the single most important item — same questions, same settings, same machine.

If this is your first time running a model on your own hardware, read [How to run AI models locally on your PC: a beginner's guide](https://ai-release.org/guides/kak-zapustit-ii-model-lokalno-na-svoem-kompyutere-gayd.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide) first; it explains the hardware side in plain language.

## Step-by-Step Instructions
![Illustration: Step-by-Step Instructions](https://ai-release.org/guides/img/kak-zapustit-novuyu-otkrytuyu-model-meta-lokalno-i-srav-en-2.jpg)


1. Install the runner. On Linux/macOS open a terminal and run:
```
curl -fsSL https://ollama.com/install.sh | sh
ollama --version
```
Expected result: a version number, e.g. `ollama version is 0.x.x`. Why it works: the script installs both the CLI and a background service that listens on port **11434**.
**Example:** on Windows, download the Ollama installer from the official site, run it, then open a new PowerShell window and type `ollama --version`.

2. Download the Meta open model.
```
ollama pull llama3.1:8b
```
Expected result: a progress bar that ends with `success`. Why it works: `pull` fetches the quantized weights into the local store, so later runs need no internet at all.
**Example:** the tag `llama3.1:8b` means "Llama 3.1 family, 8-billion-parameter size". Change the number to get a bigger or smaller variant.

3. Check that the model actually answers.
```
ollama run llama3.1:8b "In one sentence, what is a vector database?"
```
Expected result: one grammatical sentence, then the prompt returns. Why it works: `run` loads the model into memory and streams tokens back to your terminal. If you get an answer, the heavy part is done.

4. Set up a fair test file. Create `prompts.txt` with one prompt per line, covering the tasks you care about — summarising a paragraph, writing a SQL query, explaining code, translating, answering a factual question.
**Example:**
```
Summarise this in 2 sentences: <paste a paragraph>
Write a SQL query that returns the top 5 customers by revenue.
Explain what a race condition is to a junior developer.
```
Why it works: the same prompts later go to DeepSeek and Qwen, so differences you see come from the model, not from the question.

5. Run the whole file against the Meta model in a loop.
```
while IFS= read -r p; do
  echo "=== $p"
  ollama run llama3.1:8b "$p"
done < prompts.txt
```
Expected result: your prompts printed one by one, each followed by an answer. Why it works: the shell reads the file line by line and passes each line as an argument.

6. Measure speed instead of guessing. Run the same prompt twice — the first run includes loading time, the second does not.
```
ollama run llama3.1:8b --verbose "Write a haiku about caching"
```
Expected result: the answer, plus a statistics block at the bottom containing `eval rate` (tokens per second — tokens are word fragments the model generates) and `total duration`. Write those two numbers down next to the model name. That is your baseline.

7. Repeat for DeepSeek and Qwen. Pull one comparable size from each family, for example:
```
ollama pull deepseek-r1:7b
ollama pull qwen2.5:7b
```
Then re-run step 5 twice, replacing `llama3.1:8b` with each tag. Expected result: three sets of answers to the identical prompts, on the identical machine. Why it works: the runner keeps all models side by side, so switching is just a name change.

8. Serve the model over a local HTTP API, so your own scripts can use it too.
```
curl http://localhost:11434/api/generate -d '{"model":"llama3.1:8b","prompt":"Say hi","stream":false}'
```
Expected result: a JSON object whose `response` field contains the greeting. Why it works: the background service exposes an HTTP endpoint on `http://localhost:11434`; any script or client that speaks HTTP can now drive your local model.

9. Score everything in one table. Copy this skeleton into a spreadsheet or a markdown file and fill it in per prompt:
| Model | Prompt | Correct? | Speed (tokens/s) | Notes |
|---|---|---|---|---|
| llama3.1:8b | SQL top-5 customers | | | |
| deepseek-r1:7b | SQL top-5 customers | | | |
| qwen2.5:7b | SQL top-5 customers | | | |
Expected result: after 15–30 rows you can see which family wins per task type, not just "feels smarter".

10. Keep the harness. Save `prompts.txt` and the table in a repository. Next time a model drops, you swap one line and the comparison is automatic — the same discipline that saves you when a hosted model is retired, as described in [How to Survive an LLM Deprecation: Migrating Your AI Agent to a New Model](https://ai-release.org/guides/kak-perezhit-snyatie-llm-modeli-s-prodakshena-migratsiy.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide).

## Common Problems and Solutions
![Illustration: Common Problems and Solutions](https://ai-release.org/guides/img/kak-zapustit-novuyu-otkrytuyu-model-meta-lokalno-i-srav-en-3.jpg)


**1. The pull or run fails with a message about "requires more system memory than is available".**
Symptom: the download finishes, but loading the model aborts or the terminal freezes.
Solution: close heavy apps (browsers, IDEs), then pick a smaller tag (`:7b` instead of `:13b`) or a lower-bit quantization, e.g. a Q4 build instead of Q8. Check the real limit with `nvidia-smi` on NVIDIA cards or Activity Monitor on macOS. If you want to push a very large model onto consumer hardware, the tricks in [How to Run a 125B-Parameter LLM on an RTX 4090: Quantization, Offload and 100 Tokens/s](https://ai-release.org/guides/kak-zapustit-model-na-125b-parametrov-na-rtx-4090-kvant.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide) apply directly.

**2. `curl: (7) Failed to connect to localhost port 11434`.**
Symptom: the HTTP call in step 8 returns "connection refused".
Solution: the background service is not running. Type `ollama serve` in a separate terminal and leave that window open, then repeat the curl. On Windows, check the Ollama icon in the system tray; on Linux, make sure nothing else already occupies port 11434.

**3. The first run is far slower than the second.**
Symptom: `eval rate` on run one is a fraction of run two.
Solution: that is model loading, not slow inference. Always send one throwaway prompt first, then measure. Also fix the sampling settings (temperature 0, a fixed seed) — otherwise the same prompt gives different answers and your scoring table becomes noise.

## FAQ

**Do I need a dedicated graphics card to run a Meta model locally?**
No. Small quantized models run on CPU alone, just slowly. A modern GPU or Apple Silicon chip with unified memory makes them several times faster, which matters once you run dozens of test prompts.

**How much memory should I plan for?**
Enough for the model file plus its context. A 4-bit, 7–8B model fits comfortably in a machine with 8–16 GB of free memory; bigger tags need proportionally more, and if it does not fit, generation slows down instead of failing outright.

**Is comparing a local Meta model to hosted DeepSeek and Qwen fair?**
Only partly. Locally you run a compressed version, while hosted endpoints usually serve the full model at full precision. Compare what you can actually run yourself, then treat hosted results as an upper bound. For a wider view of assistant trade-offs, see [ChatGPT, Claude, Gemini, DeepSeek: AI assistant comparison](https://ai-release.org/guides/sravnenie-ii-assistentov.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide).

**Does running locally cost anything?**
No per-token fees and no API key. You pay for hardware and electricity only, and your prompts never leave your machine — which is often the real reason teams do this at all.