# How to Run a Fake LLM Server for Testing: Mock OpenAI API

> AI Release · @ai_release1 · https://ai-release.org/guides/kak-podnyat-feykovyy-llm-server-dlya-testov-mock-openai_en.html

Tests that call the real OpenAI API are slow, cost money and break the moment the network hiccups. A mock server — a tiny local program that pretends to be the OpenAI API and answers with canned JSON — fixes all three: it runs on your laptop, answers in milliseconds and never asks for a real key. This guide builds one from scratch with Python 3.8+ on macOS, Linux or Windows, and points your existing client at it with a single setting.

## Requirements and setup
![Illustration: Requirements and setup](https://ai-release.org/guides/img/kak-podnyat-feykovyy-llm-server-dlya-testov-mock-openai-en-1.jpg)

- Python 3.8 or newer, check with `python3 --version`
- `pip` for installing packages (comes with Python)
- Flask, a small Python web framework: `pip install flask`
- A terminal and a text editor
- A free local port, we use `8000` throughout
- Optional: the official OpenAI Python SDK (`pip install openai`) to test with the real client
- Optional: `curl` for manual checks (preinstalled on macOS and Linux)
- No real API key — the mock never validates one, so a dummy like `sk-test-123` is enough

Pre-flight checklist:

- [ ] `python3 --version` prints 3.8 or higher
- [ ] `pip show flask` prints a version line
- [ ] Port 8000 is not used by another process
- [ ] You know the folder where you will put the mock file

## Step-by-step instructions
![Illustration: Step-by-step instructions](https://ai-release.org/guides/img/kak-podnyat-feykovyy-llm-server-dlya-testov-mock-openai-en-2.jpg)

1. Create a project folder and a virtual environment — an isolated Python sandbox so the Flask install does not touch your system packages. Run `mkdir mock-llm && cd mock-llm`, then `python3 -m venv .venv`. Activate it: `source .venv/bin/activate` on macOS/Linux, `.venv\Scripts\activate` on Windows.

**Expected result:** your shell prompt now starts with `(.venv)`. **Why it works:** every later `pip install` lands inside this folder only.

2. Install Flask inside the environment: `pip install flask`.

**Expected result:** the last line reads something like `Successfully installed flask-3.0.0 werkzeug-3.0.1`. If you see `Requirement already satisfied`, that is fine too.

3. Create the mock server file `mock_openai_server.py` in the project folder and paste this:

```python
import os, time
from flask import Flask, request, jsonify

app = Flask(__name__)

FAIL_TIMES = int(os.environ.get("MOCK_FAIL_TIMES", "0"))
_calls = 0

@app.post("/v1/chat/completions")
def chat_completions():
    global _calls
    _calls += 1
    if _calls <= FAIL_TIMES:
        return jsonify({"error": {"message": "rate limit", "type": "rate_limit_error"}}), 429

    body = request.get_json(force=True, silent=True) or {}
    user_msg = ""
    for m in body.get("messages", []):
        if m.get("role") == "user":
            user_msg = m.get("content", "")
    reply = f"mock reply to: {user_msg}" if user_msg else "mock reply"

    return jsonify({
        "id": "chatcmpl-mock-0001",
        "object": "chat.completion",
        "created": int(time.time()),
        "model": body.get("model", "mock-model"),
        "choices": [{"index": 0,
                     "message": {"role": "assistant", "content": reply},
                     "finish_reason": "stop"}],
        "usage": {"prompt_tokens": 7, "completion_tokens": 5, "total_tokens": 12},
    })

if __name__ == "__main__":
    app.run(host="127.0.0.1", port=8000)
```

**Why it works:** the JSON shape above is what the OpenAI SDK expects — `id`, `object`, `choices[].message.content` and `usage`. Your client parses it without knowing it is fake.

4. Start the server: `python3 mock_openai_server.py`.

**Expected result:** you see `Running on http://127.0.0.1:8000`. Leave this terminal open; the server blocks it until you press Ctrl+C.

5. Test it by hand with curl in a second terminal:

```bash
curl -s http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"mock-model","messages":[{"role":"user","content":"ping"}]}'
```

**Expected result:** JSON containing `"content":"mock reply to: ping"`. If the text echoes your input, the request body is being read correctly.

6. Point your client at the mock instead of the real API. With the SDK, pass `base_url`:

```python
from openai import OpenAI

client = OpenAI(api_key="sk-test-123", base_url="http://127.0.0.1:8000/v1")
resp = client.chat.completions.create(
    model="mock-model",
    messages=[{"role": "user", "content": "ping"}],
)
print(resp.choices[0].message.content)  # mock reply to: ping
```

Or export an environment variable so your whole test run uses the mock: `export OPENAI_BASE_URL=http://127.0.0.1:8000/v1`. **Why it works:** the SDK builds URLs as `base_url + "/chat/completions"`, so the `/v1` suffix matters.

7. Add failure injection, which is the main reason to mock at all. Restart with `MOCK_FAIL_TIMES=2 python3 mock_openai_server.py`. The first two calls return HTTP 429, the third succeeds. **Expected result:** your retry logic actually runs in CI instead of being quietly skipped.

8. Mock the embeddings endpoint the same way if you test a retrieval pipeline (RAG — when a model first fetches relevant documents and only then answers from them). Add a `/v1/embeddings` route that returns a fixed vector like `[0.01, -0.02, 0.03]` for every input. For a real vector store behind it, see [Local RAG: running Qdrant or Chroma on your own mini PC](https://ai-release.org/guides/lokalnyy-rag-na-mini-pk.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide).

9. Lock the contract with one test so the mock does not drift:

```python
def test_echo():
    r = client.chat.completions.create(model="mock-model",
        messages=[{"role": "user", "content": "ping"}])
    assert r.choices[0].message.content == "mock reply to: ping"
```

**Expected result:** `1 passed` in under a second. That speed is the whole payoff — no network, no quota.

## Possible problems and solutions
![Illustration: Possible problems and solutions](https://ai-release.org/guides/img/kak-podnyat-feykovyy-llm-server-dlya-testov-mock-openai-en-3.jpg)

**Problem 1: the client hangs or prints `Connection refused` / `ECONNREFUSED 127.0.0.1:8000`.**
Symptom: curl returns `curl: (7) Failed to connect`. The server process is not running, crashed, or binds a different port. Fix: check the first terminal for a traceback, confirm the line `Running on http://127.0.0.1:8000`, and start `python3 mock_openai_server.py` again. If you changed the port in code, change the URL in your client too.

**Problem 2: `Error code: 404 - Not Found` with a valid-looking request.**
Symptom: the mock is up but every call 404s. Two usual causes: your `base_url` is missing `/v1` (so the SDK requests `/chat/completions`), or you call `/v1/completions` while the mock only defines `/v1/chat/completions`. Fix: set `base_url="http://127.0.0.1:8000/v1"` and add the missing route — the decorator string must match the URL exactly.

**Problem 3: requests still reach the real api.openai.com.**
Symptom: tests are slow and you see real token usage on your account. The environment variable was exported in a different shell, or the client passes an explicit base URL that overrides it. Fix: run `echo $OPENAI_BASE_URL` in the same terminal as the test, then re-export it, and check the client constructor for a hard-coded default.

**Problem 4: `400 Bad Request` and the reply is empty.**
Symptom: the mock answers but `content` is `mock reply` with no echo. The body was not parsed. Fix: always send `-H "Content-Type: application/json"` in curl, and keep `request.get_json(force=True, silent=True)` in the handler so a malformed body returns an empty dict instead of raising.

## FAQ

**Does the SDK need a real API key to talk to my mock?**
No. The SDK only puts the key in an `Authorization: Bearer ...` header. Your mock ignores headers, so `sk-test-123` works. Keep the variable name the same (`OPENAI_API_KEY`) so no real code changes are needed.

**Why does streaming work in production but not against the mock?**
You probably only implemented the non-streaming route. Streaming uses server-sent events: each chunk is a line `data: {...}` followed by a blank line, and the stream ends with `data: [DONE]`. Flask can return `Response(generator(), mimetype="text/event-stream")` with those lines.

**Can I run this mock in Docker or CI?**
Yes. Put Flask and your file in an image, expose port 8000, and set `OPENAI_BASE_URL` to the service address before the tests run. `MOCK_FAIL_TIMES` is a plain environment variable, so the same image covers happy-path and retry tests.

**How is this different from mocking a tool-calling or MCP server?**
A mock API answers HTTP with fixed JSON; MCP — a protocol for plugging tools and data sources into an LLM — needs a protocol-level handshake, not just a JSON body. If that is what you are testing, start from [MCP explained: what it is and how to connect your first MCP server](https://ai-release.org/guides/mcp-chto-eto-i-kak-podklyuchit-server.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide).