# How many tools should an MCP server expose? Measured from 5 to 100

> AI Release · @ai_release1 · https://ai-release.org/guides/skolko-instrumentov-u-mcp-servera-zamery_en.html

## TL;DR

- **Time to complete:** 60 minutes
- **Difficulty:** Intermediate
- **You will need:** Linux, macOS, or WSL2; Python 3.10+ (`python3 --version`); the `mcp` and `openai` Python packages installed from their current releases (`pip show mcp` prints the version you actually have); an API key and base URL for an OpenAI-compatible endpoint; about 200 MB of free disk space for the virtualenv and results.
- **What you get:** a parameterised MCP server that exposes exactly N tools, a benchmark that runs it at N = 5, 10, 25, 50, 100 and writes payload size, input tokens, `tools/list` latency, and tool-selection accuracy to `results.csv`. The numbers in the table are the ones *you* measure. Published per-model results are not reproduced here: no measurement data was supplied with this material, so treat every model-specific figure as “check the documentation”.

## Requirements and setup
![Illustration: Requirements and setup](https://ai-release.org/guides/img/skolko-instrumentov-u-mcp-servera-zamery-en-1.jpg?v=1793343153)


An **MCP server** (Model Context Protocol — an open protocol that lets an LLM client call functions exposed by a separate process) publishes a list of **tools**. Each tool is sent to the model as a name, a description, and a JSON Schema of its arguments. That list travels inside every request, so the tool count sets two costs at once: input tokens (money and context-window space) and selection difficulty (the chance the model calls the wrong tool). The benchmark below separates those two effects instead of guessing at them.

Data flow you are about to build:

```
questions.jsonl ──► bench.py ──► MCP server (env TOOL_COUNT=N)
                        │                 │
                        │           tools/list → name + description + inputSchema
                        ▼                 ▼
              usage.prompt_tokens + chosen tool name ──► results.csv
```

- Linux, macOS, or WSL2 on Windows 10/11
- Python 3.10 or newer, with `venv` available
- An OpenAI-compatible endpoint: `LLM_BASE_URL`, `LLM_API_KEY`, `LLM_MODEL` (the exact model ID from your provider's model list — see [Best AI models and neural networks in 2026](https://ai-release.org/guides/luchshie-nejroseti-i-modeli-2026.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide) if you have not picked one)
- If you plan to run the selection model locally instead of over an API, size the machine first with the [RAM and VRAM table](https://ai-release.org/guides/localnye-ii-modeli-skolko-ram-i-vram.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide)
- Confirm your model's context window and any documented cap on tools per request — **check the documentation**, values differ per provider and are not assumed here

Checklist before you start:

- [ ] `python3 --version` prints 3.10 or higher
- [ ] `pip show mcp openai` returns both packages
- [ ] `echo $LLM_BASE_URL` prints a URL ending in `/v1`
- [ ] `echo $LLM_MODEL` prints a model ID that exists in your provider's list
- [ ] Your account has quota left for ~150 short requests (5 runs × 30 questions)

```bash
python3 -m venv .venv && source .venv/bin/activate
pip install --upgrade pip
pip install mcp openai
python -c "import mcp, openai; print('ok')"
```

Expected output: `ok`. If the import fails, the package did not install into the active virtualenv — re-run with `.venv/bin/pip install mcp openai`.

## Step 1. Build a server whose tool count is a variable

**What to do:** create `server.py` that reads `TOOL_COUNT` from the environment and registers exactly that many tools, and `inspect_tools.py` that starts it over stdio and prints how many tools came back plus the byte size of their definitions.

**Why:** a fixed server cannot be swept from 5 to 100 tools. Making the count an environment variable means one code path, one process, five configurations — so a difference in the results is caused by the count, not by different code.

**How to check:** `inspect_tools.py` must print one line per configuration with `tools=` equal to the requested N.

```python
# server.py
import os
from mcp.server.fastmcp import FastMCP

TOOL_COUNT = int(os.environ.get("TOOL_COUNT", "5"))
mcp = FastMCP("tool-count-bench")

def build(i: int):
    def handler(query: str) -> str:
        return f"shard-{i:03d}: no records for '{query}'"
    handler.__name__ = f"search_shard_{i:03d}"
    handler.__doc__ = (
        f"Search documentation shard {i:03d}. "
        f"Call this only when the question names shard {i:03d}."
    )
    return handler

for i in range(TOOL_COUNT):
    mcp.tool()(build(i))

if __name__ == "__main__":
    mcp.run(transport="stdio")   # stdio = JSON-RPC over stdin/stdout
```

```python
# inspect_tools.py
import asyncio, json, os, sys, time
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client

async def inspect(n: int):
    params = StdioServerParameters(
        command=sys.executable,
        args=["server.py"],
        env={**os.environ, "TOOL_COUNT": str(n)},
    )
    async with stdio_client(params) as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()
            t0 = time.perf_counter()
            tools = (await session.list_tools()).tools
            dt = (time.perf_counter() - t0) * 1000
            payload = json.dumps([t.model_dump(exclude_none=True) for t in tools])
            print(f"n={n:>3} tools={len(tools):>3} bytes={len(payload):>6} list_ms={dt:.1f}")

async def main():
    for n in (5, 10, 25, 50, 100):
        await inspect(n)

asyncio.run(main())
```

```bash
python inspect_tools.py
```

Example of the output format (your byte counts will differ — these are placeholders, not measured results):

```
n=  5 tools=  5 bytes=  1900 list_ms=  40.0
n= 10 tools= 10 bytes=  3800 list_ms=  42.0
n=100 tools=100 bytes= 38000 list_ms=  90.0
```

Why it works: `TOOL_COUNT` is read at import time, so each subprocess registers a different number of handlers, and `tools/list` returns the live registry — the byte size grows because every extra tool adds its name, description, and argument schema.

## Step 2. Measure tokens, latency, and correct tool choice at each N
![Illustration: Step 2. Measure tokens, latency, and correct tool choice at each N](https://ai-release.org/guides/img/skolko-instrumentov-u-mcp-servera-zamery-en-2.jpg?v=1793343163)


**What to do:** generate a labelled question set, convert the MCP tool definitions into the OpenAI tool format, call the model once per question with `tool_choice="auto"`, and log the chosen tool name plus `usage.prompt_tokens`.

**Why:** byte size is a proxy. The number that hits your invoice and your context window is `prompt_tokens` from the response, and the number that matters for reliability is how often the model picks the intended tool. Measuring both at the same N tells you whether a failure is a budget problem or a selection problem.

**How to check:** `results.csv` gains one row per (N, metric) and every accuracy value is between 0 and 100.

```python
# bench.py
import asyncio, csv, json, os, sys
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
from openai import OpenAI

client = OpenAI(base_url=os.environ["LLM_BASE_URL"], api_key=os.environ["LLM_API_KEY"])
MODEL = os.environ["LLM_MODEL"]

async def fetch_tools(n: int):
    params = StdioServerParameters(command=sys.executable, args=["server.py"],
                                   env={**os.environ, "TOOL_COUNT": str(n)})
    async with stdio_client(params) as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()
            return [t.model_dump(exclude_none=True) for t in (await session.list_tools()).tools]

def to_openai(tools):
    return [{"type": "function",
             "function": {"name": t["name"],
                          "description": t.get("description", ""),
                          "parameters": t["inputSchema"]}} for t in tools]

def pick(tools, question: str):
    resp = client.chat.completions.create(
        model=MODEL, tools=tools, tool_choice="auto", temperature=0,
        messages=[{"role": "user", "content": question}],
    )
    calls = resp.choices[0].message.tool_calls or []
    return (calls[0].function.name if calls else None), resp.usage.prompt_tokens

async def run(n: int, rows: list):
    tools = to_openai(await fetch_tools(n))
    available = {t["function"]["name"] for t in tools}
    prompt_tokens, correct, total = 0, 0, 0
    for name in sorted(available):
        question = f"What does {name.replace('search_shard_', 'shard ')} contain?"
        chosen, used = pick(tools, question)
        prompt_tokens, total = used, total + 1
        correct += int(chosen == name)
    rows.append({"tools": n, "prompt_tokens": prompt_tokens,
                 "correct": correct, "total": total,
                 "accuracy_pct": round(100 * correct / total, 1)})
    print(rows[-1])

async def main():
    rows: list = []
    for n in (5, 10, 25, 50, 100):
        await run(n, rows)
    with open("results.csv", "w", newline="") as f:
        w = csv.DictWriter(f, fieldnames=rows[0].keys()); w.writeheader(); w.writerows(rows)

asyncio.run(main())
```

```bash
export LLM_BASE_URL="https://api.example.com/v1"
export LLM_API_KEY="<your-key>"
export LLM_MODEL="<your-model-id>"
python bench.py
```

`<your-model-id>` must be replaced with an ID that your provider actually serves; `<your-key>` with the key from your account. Why it works: `prompt_tokens` is counted by the server after the tools array is serialised, so it includes the tool definitions — that is the real overhead of running N tools. Because each question names exactly one shard, the expected tool is unambiguous and the accuracy figure is a fair score.

## Step 3. Read the table, pick a count, and verify the choice
![Illustration: Step 3. Read the table, pick a count, and verify the choice](https://ai-release.org/guides/img/skolko-instrumentov-u-mcp-servera-zamery-en-3.jpg?v=1793343170)


**What to do:** fill in the comparison table from `results.csv`, find the largest N that still meets your accuracy target, then re-run that single N and confirm the numbers reproduce.

**Why:** a single run can be noise. Reproducing the chosen configuration once more is what turns a number into a decision — and it is the number you will quote when someone asks why the server is split.

**How to check:** the second run of the chosen N lands within a few percentage points of the first, and `results.csv` has five rows.

| Tools | Definition bytes | Input tokens (prompt) | tools/list latency, ms | Correct tool choice, % |
|---|---|---|---|---|
| 5 | from `inspect_tools.py` | from `results.csv` | from `inspect_tools.py` | from `results.csv` |
| 10 | … | … | … | … |
| 25 | … | … | … | … |
| 50 | … | … | … | … |
| 100 | … | … | … | … |

```bash
column -s, -t results.csv
TOOL_COUNT=25 python inspect_tools.py   # reproduce the chosen point
```

Expected: the table prints aligned columns, and the second `inspect_tools.py` line shows `tools=25` with a byte count close to the value already in your table.

Decision rules, applied to your own numbers:

1. If accuracy is flat from N to the next step but tokens jump, stop at the lower N — the extra tools buy nothing.
2. If accuracy falls at N but tokens are still small, the problem is description quality, not count: make each description state *when* to call the tool, as in Step 1.
3. If tokens approach the context window before accuracy drops, split the server by domain (for example `search-*` in one server, `write-*` in another) and connect only the relevant one per session.
4. Record the chosen N in the repository README next to the model ID — the ceiling is model-specific, so it must be re-measured when the model changes.

## Troubleshooting

**Problem. The client crashes while parsing the first message from the server.**
- Ошибка: `json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)`
- Причина: something in `server.py` writes to stdout. Under the stdio transport, stdout *is* the JSON-RPC channel, so a stray `print("started")` or a library banner corrupts the stream.
- Решение: send diagnostics to stderr instead and re-check that stdout stays clean:

```bash
sed -i 's/print(/print(file=sys.stderr, /' server.py   # or delete the prints
TOOL_COUNT=5 python server.py 1>/dev/null   # nothing should appear on the terminal
```

**Problem. The run dies at 50 or 100 tools.**
- Ошибка: `openai.BadRequestError: Error code: 400 - {'error': {'code': 'context_length_exceeded', 'message': "This model's maximum context length is <context> tokens. However, your messages resulted in <total> tokens."}}` (the two numbers come from your model — **check the documentation** for the exact limit)
- Причина: the serialised tool definitions plus the question exceed the window. Bytes scale roughly linearly with N, so the break point is predictable from the `bytes=` column.
- Решение: shorten descriptions to one sentence, drop unused parameters from the schemas, or split into two servers and re-run the sweep per server:

```bash
TOOL_COUNT=50 python bench.py   # find the largest N that fits before splitting
```

**Problem. The sweep stops after the first configuration.**
- Ошибка: `RuntimeError: Attempted to exit cancel scope in a different task than it was entered in`
- Причина: `asyncio.run()` was called once per N, so the stdio session is torn down outside the task that created it. The anyio task group behind `stdio_client` refuses that teardown.
- Решение: drive all five configurations from one event loop, exactly as `bench.py` and `inspect_tools.py` above do (`async def main()` looping over N, a single `asyncio.run(main())`), and install the current `mcp` release with `pip install --upgrade mcp`. Verify with `python inspect_tools.py`: five lines, no traceback.

## Summary and next steps

You now have a reproducible sweep that turns “how many tools should the MCP server expose” into a table of tokens, latency, and accuracy for your own model. Next, cut the tools that never win a question, then re-run the same harness to confirm accuracy held while `prompt_tokens` fell.

## FAQ

**What is an MCP server in one sentence?**
A separate process that publishes tools — name, description, and argument schema — which an LLM client can list and call over a standard protocol, so the model reaches your code without custom glue per integration.

**How many tools should one MCP server expose?**
There is no universal number: the ceiling depends on the model's context window and on how distinguishable your descriptions are. Run the sweep above and take the largest N that still meets your accuracy target — the answer is a measurement, not a constant.

**Do tool descriptions really cost money?**
Yes. They are serialised into the request, so they are billed as input tokens; the `prompt_tokens` field in `results.csv` is that cost measured directly. Shortening a description across 100 tools removes that text from every single request.

**Can I keep 100 tools and still get good selection?**
Sometimes, if each description states precisely when to call it and the schemas stay small. If accuracy still collapses at 100, split by domain or expose a single router tool that dispatches to the rest — then re-measure with the same harness rather than assuming the fix worked.