# How to deploy Kimi K3 on Cerebras: step-by-step setup and comparison with local hosting

> AI Release · @ai_release1 · https://ai-release.org/guides/kak-razvernut-kimi-k3-na-cerebras-poshagovaya-nastroyka_en.html

## TL;DR

- Time to complete: 20 minutes
- Difficulty: Intermediate
- You will need: a Cerebras account with an API key, `curl`, Python 3.9+ with the `openai` package, a terminal (bash, zsh, PowerShell 7 or WSL2). Optional: Ollama if you want the local-run comparison in Step 3.
- Result: a working `kimi` request against the Cerebras hosted endpoint, a repeatable latency/tokens-per-second measurement, and a side-by-side table you can fill with your own numbers.
- Not covered by this guide (verify in the Cerebras and Moonshot documentation before you plan capacity): the exact model ID string currently served, context window, per-token price and rate limits.

Two terms used below. **Hosted inference endpoint** — an HTTP API where someone else runs the model on their hardware and you pay per token. **OpenAI-compatible API** — an HTTP interface that copies OpenAI's request and response shape, so the official `openai` SDK works unchanged once you point `base_url` at another host.

## Requirements and setup
![Illustration: Requirements and setup](https://ai-release.org/guides/img/kak-razvernut-kimi-k3-na-cerebras-poshagovaya-nastroyka-en-1.jpg?v=1792962289)


- **OS:** Linux, macOS, Windows 11 with PowerShell 7, or WSL2. The commands below are shown for bash; PowerShell equivalents are given where the syntax differs.
- **Python:** 3.9 or newer. Check with `python3 --version`.
- **SDK:** the `openai` Python package, any 1.x release (the `base_url` argument exists in all of them). Install and confirm:

```bash
python3 -m pip install --upgrade openai
python3 -m pip show openai | head -n 2
```

Expected output starts with `Name: openai` and `Version: 1.` — if you see `0.` you have a pre-1.0 build that does not accept `base_url`.

- **curl:** `curl --version`. On Windows use `curl.exe`, which ships with Windows 11.
- **API key:** created in the Cerebras console. Store it in an environment variable, never in source code.
- **Disk:** about 1 GB free only if you also install Ollama for the local comparison.
- **Access check:** confirm that the Kimi model you intend to call is enabled for your account — model availability differs per plan. Use the model-list call in Step 1 rather than guessing the ID.

**Example:** export the key for the current shell session:

```bash
export CEREBRAS_API_KEY="csk-<your-key>"
```

```powershell
$env:CEREBRAS_API_KEY = "csk-<your-key>"
```

Why this works: the SDK reads `CEREBRAS_API_KEY` from the process environment, so the secret never lands in `bench_kimi.py` or in git.

## Step 1. Create the API key and confirm the model ID

**What to do:** list the models your key can reach, then copy the exact ID of the Kimi entry into a variable.

```bash
curl -s https://api.cerebras.ai/v1/models \
  -H "Authorization: Bearer $CEREBRAS_API_KEY" | python3 -m json.tool
```

**Why:** the `model` field in every request must match a served ID byte for byte. IDs change between releases, and a typo costs you a 404 later, in the middle of a longer script.

**How to check:** the response is JSON with a `data` array. Each element has an `id` field. Find the one that names Kimi and save it:

```bash
export KIMI_MODEL_ID="<kimi-model-id>"
echo "$KIMI_MODEL_ID"
```

Expected: the shell echoes the exact string, for example a single token with no spaces. If `data` contains no Kimi entry, the model is not enabled for your account — that is a plan question for the console, not a code problem. Base URL, model ID and pricing are the three values you should re-read in the [Kimi K3.1 Pro API walkthrough](https://ai-release.org/guides/kak-podklyuchit-kimi-k3-1-pro-cherez-api-klyuchi-tseny.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide) whenever you switch accounts.

**Example:** a truncated list looks like this — note that only the `id` values matter here:

```json
{
  "data": [
    { "id": "some-model-a", "object": "model" },
    { "id": "<kimi-model-id>", "object": "model" }
  ],
  "object": "list"
}
```

## Step 2. Send your first request over the OpenAI-compatible endpoint
![Illustration: Step 2. Send your first request over the OpenAI-compatible endpoint](https://ai-release.org/guides/img/kak-razvernut-kimi-k3-na-cerebras-poshagovaya-nastroyka-en-2.jpg?v=1792962294)


**What to do:** make one raw HTTP call with curl to prove the key and the model ID work together, then repeat it from Python.

```bash
curl -s https://api.cerebras.ai/v1/chat/completions \
  -H "Authorization: Bearer $CEREBRAS_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"$KIMI_MODEL_ID\",
    \"messages\": [{\"role\": \"user\", \"content\": \"Explain in one sentence what an inference endpoint is.\"}],
    \"max_tokens\": 64
  }" | python3 -m json.tool
```

**Why:** curl isolates the network problem from the SDK problem. If this call returns text, every later failure is in your Python code, not in the credentials.

**How to check:** look for three keys in the JSON: `choices[0].message.content` (the answer text), `usage.prompt_tokens` and `usage.completion_tokens`. Those usage fields are what you will bill against and measure with.

**Example:** a healthy response body, shortened:

```json
{
  "id": "chatcmpl-...",
  "model": "<kimi-model-id>",
  "choices": [{ "index": 0, "message": { "role": "assistant", "content": "An inference endpoint is an HTTP service that runs a trained model and returns its output." }, "finish_reason": "stop" }],
  "usage": { "prompt_tokens": 18, "completion_tokens": 22, "total_tokens": 40 }
}
```

Now the same request from Python. Save this as `bench_kimi.py`:

```python
import os, time
from openai import OpenAI

client = OpenAI(
    base_url="https://api.cerebras.ai/v1",
    api_key=os.environ["CEREBRAS_API_KEY"],
    timeout=30.0,
)

start = time.perf_counter()
resp = client.chat.completions.create(
    model=os.environ["KIMI_MODEL_ID"],
    messages=[{"role": "user", "content": "List three checks to run before shipping an LLM feature."}],
    max_tokens=200,
    temperature=0.2,
)
elapsed = time.perf_counter() - start

print(resp.choices[0].message.content)
print(f"total={elapsed:.2f}s completion_tokens={resp.usage.completion_tokens} "
      f"tok_per_s={resp.usage.completion_tokens / elapsed:.1f}")
```

Why this works: `base_url` swaps the host, and the SDK keeps building the same JSON body it would send to OpenAI. Only `model`, `messages` and the token limits are yours to set.

**Example run:**

```bash
python3 bench_kimi.py
```

```text
1. Pin the model ID and record the date you validated it.
2. Log prompt and completion tokens per request.
3. Add a timeout and a fallback path.
total=1.24s completion_tokens=48 tok_per_s=38.7
```

If you would rather not manage the HTTP layer at all, the key creation and billing flow in [OpenAI models via Google Cloud](https://ai-release.org/guides/kak-podklyuchit-modeli-openai-cherez-google-cloud-posha.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide) shows the managed-gateway alternative — useful if your organisation already standardises on one cloud vendor's billing.

## Step 3. Verify output, measure latency, and compare with a local run

**What to do:** run the same prompt three times against the hosted endpoint, then — optionally — against a local model through Ollama's OpenAI-compatible port on `127.0.0.1:11434`, and fill in the table with your own numbers.

```bash
for i in 1 2 3; do python3 bench_kimi.py | tail -n 1; done
```

**Why:** one measurement is noise. Three runs show whether your `tok_per_s` figure is stable or whether the tail of the distribution is what your users actually hit.

**How to check:** time-to-first-token is the number users feel, and the script above measures total time only. Add a streaming variant to capture it:

```python
import os, time
from openai import OpenAI

client = OpenAI(base_url="https://api.cerebras.ai/v1", api_key=os.environ["CEREBRAS_API_KEY"])
stream = client.chat.completions.create(
    model=os.environ["KIMI_MODEL_ID"],
    messages=[{"role": "user", "content": "Write one paragraph about API retries."}],
    max_tokens=200, stream=True,
)
start = first = None
for chunk in stream:
    if not chunk.choices or not chunk.choices[0].delta.content:
        continue
    if first is None:
        first = time.perf_counter()
        print(f"TTFT={first - start:.2f}s")
print("done")
```

Local baseline, if you want one:

```bash
ollama serve &
ollama pull <local-model-tag>
```

Then point the same `bench_kimi.py` at `base_url="http://localhost:11434/v1"` with `api_key="ollama"` and `model="<local-model-tag>"`. The dummy key is required by the SDK, not by the server. Wiring a local model into automation is covered end to end in [n8n with a local LLM via Ollama](https://ai-release.org/guides/kak-podklyuchit-n8n-k-lokalnoy-yazykovoy-modeli-poshago.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide).

**Example comparison table** — fill the middle column with the numbers your script prints:

| Check | Hosted Cerebras endpoint | Local run (Ollama) |
|---|---|---|
| Where the weights live | Provider's infrastructure | Your disk, after `ollama pull` |
| What you pay for | Tokens, per provider pricing | Your own hardware and electricity |
| Network required at request time | Yes | No, traffic stays on `127.0.0.1` |
| Measured `tok_per_s` | `<paste from bench_kimi.py>` | `<paste from bench_kimi.py>` |
| Measured TTFT | `<paste from stream script>` | `<paste from stream script>` |
| Model ID to pin | `<kimi-model-id>` | `<local-model-tag>` |

Verification checklist before you call the setup done:

- [ ] `curl` to `/v1/models` returns `200` and lists the Kimi ID
- [ ] `bench_kimi.py` prints an answer and a `tok_per_s` line
- [ ] the streaming script prints `TTFT=` and `done`
- [ ] three consecutive runs complete without a retry
- [ ] the model ID and the date are written into your config, not your notes

## Troubleshooting

**Problem. Every request fails immediately with 401**

- Error: `Error code: 401 - {'message': 'Invalid API key'}` (the wording is produced by the server; the `401` status is the reliable signal)
- Cause: the variable was exported in a different shell, contains a trailing newline or quotes, or you copied a key from a deleted project.
- Solution: print the key length, not the key itself, and re-export in the same terminal that runs the script.

```bash
echo -n "$CEREBRAS_API_KEY" | wc -c
```

If the count includes a blank line, re-export with `export CEREBRAS_API_KEY="csk-..."` on one line and rerun the curl call from Step 2.

**Problem. The request reaches the server but no answer comes back**

- Error: `Error code: 404 - {'message': 'Model not found', 'code': 'model_not_found'}`
- Cause: `KIMI_MODEL_ID` is unset, contains the display name instead of the ID, or the model is not enabled for your account.
- Solution: rerun `echo "$KIMI_MODEL_ID"` — an empty line means the variable never survived a new terminal. Re-export it from the `/v1/models` output, then check the console for model access.

**Problem. Long prompts die while short ones succeed**

- Error: `openai.APITimeoutError: Request timed out.`
- Cause: the default client timeout is short relative to a long generation, and non-streaming calls wait for the full answer before returning anything.
- Solution: raise the timeout and stream instead of blocking, so bytes keep arriving.

```python
client = OpenAI(base_url="https://api.cerebras.ai/v1", api_key=os.environ["CEREBRAS_API_KEY"], timeout=120.0)
```

## Summary and next steps

You now have a pinned model ID, a working OpenAI-compatible client and a repeatable measurement script you can rerun after every provider change. Next, move the key into a secret manager and add the three-run latency check to CI so a regression shows up before your users find it.

## FAQ

**Is the Cerebras endpoint free to try?**
Check the current plan and pricing page in the Cerebras console — free tiers and trial credits change, so no fixed answer here would stay correct.

**Can I use the official OpenAI Python SDK instead of raw HTTP?**
Yes. Set `base_url="https://api.cerebras.ai/v1"`, pass your key as `api_key`, and set `model` to the exact ID from `/v1/models`.

**What is the context window of Kimi K3 on Cerebras?**
Verify it in the provider documentation for the exact model ID you pinned; limits differ per model release and per plan, and a wrong assumption only surfaces when a long prompt is truncated.

**When should I run the model locally instead of on Cerebras?**
Choose local when data cannot leave your machine or you need offline operation, and accept that you supply the GPU memory and ops work yourself. Choose the hosted endpoint when you want the model available without provisioning hardware.