How to Run a Fake LLM Server for Testing: Mock OpenAI API
Tests that call the real OpenAI API are slow, cost money and break the moment the network hiccups. A mock server — a tiny local program that pretends to be the OpenAI API and answers with canned JSON — fixes all three: it runs on your laptop, answers in milliseconds and never asks for a real key. This guide builds one from scratch with Python 3.8+ on macOS, Linux or Windows, and points your existing client at it with a single setting.
Requirements and setup
- Python 3.8 or newer, check with `python3 --version`
- `pip` for installing packages (comes with Python)
- Flask, a small Python web framework: `pip install flask`
- A terminal and a text editor
- A free local port, we use `8000` throughout
- Optional: the official OpenAI Python SDK (`pip install openai`) to test with the real client
- Optional: `curl` for manual checks (preinstalled on macOS and Linux)
- No real API key — the mock never validates one, so a dummy like `sk-test-123` is enough
Pre-flight checklist:
- [ ] `python3 --version` prints 3.8 or higher
- [ ] `pip show flask` prints a version line
- [ ] Port 8000 is not used by another process
- [ ] You know the folder where you will put the mock file
Step-by-step instructions
1. Create a project folder and a virtual environment — an isolated Python sandbox so the Flask install does not touch your system packages. Run `mkdir mock-llm && cd mock-llm`, then `python3 -m venv .venv`. Activate it: `source .venv/bin/activate` on macOS/Linux, `.venv\Scripts\activate` on Windows.
Expected result:your shell prompt now starts with `(.venv)`. **Why it works:** every later `pip install` lands inside this folder only.
2. Install Flask inside the environment: `pip install flask`.
Expected result:the last line reads something like `Successfully installed flask-3.0.0 werkzeug-3.0.1`. If you see `Requirement already satisfied`, that is fine too.
3. Create the mock server file `mock_openai_server.py` in the project folder and paste this:
import os, time
from flask import Flask, request, jsonify
app = Flask(__name__)
FAIL_TIMES = int(os.environ.get("MOCK_FAIL_TIMES", "0"))
_calls = 0
@app.post("/v1/chat/completions")
def chat_completions():
global _calls
_calls += 1
if _calls <= FAIL_TIMES:
return jsonify({"error": {"message": "rate limit", "type": "rate_limit_error"}}), 429
body = request.get_json(force=True, silent=True) or {}
user_msg = ""
for m in body.get("messages", []):
if m.get("role") == "user":
user_msg = m.get("content", "")
reply = f"mock reply to: {user_msg}" if user_msg else "mock reply"
return jsonify({
"id": "chatcmpl-mock-0001",
"object": "chat.completion",
"created": int(time.time()),
"model": body.get("model", "mock-model"),
"choices": [{"index": 0,
"message": {"role": "assistant", "content": reply},
"finish_reason": "stop"}],
"usage": {"prompt_tokens": 7, "completion_tokens": 5, "total_tokens": 12},
})
if __name__ == "__main__":
app.run(host="127.0.0.1", port=8000)Why it works:the JSON shape above is what the OpenAI SDK expects — `id`, `object`, `choices[].message.content` and `usage`. Your client parses it without knowing it is fake.
4. Start the server: `python3 mock_openai_server.py`.
Expected result:you see `Running on http://127.0.0.1:8000`. Leave this terminal open; the server blocks it until you press Ctrl+C.
5. Test it by hand with curl in a second terminal:
curl -s http://127.0.0.1:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"mock-model","messages":[{"role":"user","content":"ping"}]}'Expected result:JSON containing `"content":"mock reply to: ping"`. If the text echoes your input, the request body is being read correctly.
6. Point your client at the mock instead of the real API. With the SDK, pass `base_url`:
from openai import OpenAI
client = OpenAI(api_key="sk-test-123", base_url="http://127.0.0.1:8000/v1")
resp = client.chat.completions.create(
model="mock-model",
messages=[{"role": "user", "content": "ping"}],
)
print(resp.choices[0].message.content) # mock reply to: pingOr export an environment variable so your whole test run uses the mock: `export OPENAI_BASE_URL=http://127.0.0.1:8000/v1`. **Why it works:** the SDK builds URLs as `base_url + "/chat/completions"`, so the `/v1` suffix matters.
7. Add failure injection, which is the main reason to mock at all. Restart with `MOCK_FAIL_TIMES=2 python3 mock_openai_server.py`. The first two calls return HTTP 429, the third succeeds. **Expected result:** your retry logic actually runs in CI instead of being quietly skipped.
8. Mock the embeddings endpoint the same way if you test a retrieval pipeline (RAG — when a model first fetches relevant documents and only then answers from them). Add a `/v1/embeddings` route that returns a fixed vector like `[0.01, -0.02, 0.03]` for every input. For a real vector store behind it, see Local RAG: running Qdrant or Chroma on your own mini PC.
9. Lock the contract with one test so the mock does not drift:
def test_echo():
r = client.chat.completions.create(model="mock-model",
messages=[{"role": "user", "content": "ping"}])
assert r.choices[0].message.content == "mock reply to: ping"Expected result:`1 passed` in under a second. That speed is the whole payoff — no network, no quota.
Possible problems and solutions
Problem 1: the client hangs or prints `Connection refused` / `ECONNREFUSED 127.0.0.1:8000`.
Symptom: curl returns `curl: (7) Failed to connect`. The server process is not running, crashed, or binds a different port. Fix: check the first terminal for a traceback, confirm the line `Running on http://127.0.0.1:8000`, and start `python3 mock_openai_server.py` again. If you changed the port in code, change the URL in your client too.
Problem 2: `Error code: 404 - Not Found` with a valid-looking request.
Symptom: the mock is up but every call 404s. Two usual causes: your `base_url` is missing `/v1` (so the SDK requests `/chat/completions`), or you call `/v1/completions` while the mock only defines `/v1/chat/completions`. Fix: set `base_url="http://127.0.0.1:8000/v1"` and add the missing route — the decorator string must match the URL exactly.
Problem 3: requests still reach the real api.openai.com.
Symptom: tests are slow and you see real token usage on your account. The environment variable was exported in a different shell, or the client passes an explicit base URL that overrides it. Fix: run `echo $OPENAI_BASE_URL` in the same terminal as the test, then re-export it, and check the client constructor for a hard-coded default.
Problem 4: `400 Bad Request` and the reply is empty.
Symptom: the mock answers but `content` is `mock reply` with no echo. The body was not parsed. Fix: always send `-H "Content-Type: application/json"` in curl, and keep `request.get_json(force=True, silent=True)` in the handler so a malformed body returns an empty dict instead of raising.
FAQ
Does the SDK need a real API key to talk to my mock?
No. The SDK only puts the key in an `Authorization: Bearer ...` header. Your mock ignores headers, so `sk-test-123` works. Keep the variable name the same (`OPENAI_API_KEY`) so no real code changes are needed.
Why does streaming work in production but not against the mock?
You probably only implemented the non-streaming route. Streaming uses server-sent events: each chunk is a line `data: {...}` followed by a blank line, and the stream ends with `data: [DONE]`. Flask can return `Response(generator(), mimetype="text/event-stream")` with those lines.
Can I run this mock in Docker or CI?
Yes. Put Flask and your file in an image, expose port 8000, and set `OPENAI_BASE_URL` to the service address before the tests run. `MOCK_FAIL_TIMES` is a plain environment variable, so the same image covers happy-path and retry tests.
How is this different from mocking a tool-calling or MCP server?
A mock API answers HTTP with fixed JSON; MCP — a protocol for plugging tools and data sources into an LLM — needs a protocol-level handshake, not just a JSON body. If that is what you are testing, start from MCP explained: what it is and how to connect your first MCP server.
🤖 AI summary
- Updated: 2026-10-14
- Topics: mock llm server, fake openai api для тестов, dummy llm, тестирование llm-приложений без api
- Summary: Руководство по локальному dummy-серверу, который отвечает по шаблону или повторяет запрос: экономит токены и деньги при отладке чат-ботов, агентов и стриминга.
- Key facts:
- Tests that call the real OpenAI API are slow, cost money and break the moment the network hiccups.
- A mock server — a tiny local program that pretends to be the OpenAI API and answers with canned JSON — fixes all three: it runs on your laptop, answers in milliseconds and never asks for a real key.
- This guide builds one from scratch with Python 3.8+ on macOS, Linux or Windows, and points your existing client at it with a single setting.
- Requirements and setup Illustration: Requirements and setup Python 3.8 or newer, check with python3 --version pip for installing packages (comes with Python) Flask, a small Python web framework: pip install flask A terminal and a text editor A free local port, we use 8000 throughout Optional: the official OpenAI Python SDK ( pip install openai ) to test with the real client Optional: curl for manual checks (preinstalled on macOS and Linux) No real API key — the mock never validates one, so a dummy like sk-test-123 is enough Pre-flight checklist: [ ] python3 --version prints 3.8 or higher [ ] pip show flask prints a version line [ ] Port 8000 is not used by another process [ ] You know the folder where you will put the mock file Step-by-step instructions Illustration: Step-by-step instructions 1.
- Create a project folder and a virtual environment — an isolated Python sandbox so the Flask install does not touch your system packages.
- Run mkdir mock-llm && cd mock-llm , then python3 -m venv .venv .