← All guides AI Release
RU Subscribe to @ai_release1

How many tools should an MCP server expose? Measured from 5 to 100

Updated: 30.10.2026 · AI Release · @ai_release1
How many tools should an MCP server expose? Measured from 5 to 100

TL;DR

Requirements and setup

Illustration: Requirements and setup
Illustration: Requirements and setup

An MCP server (Model Context Protocol — an open protocol that lets an LLM client call functions exposed by a separate process) publishes a list of tools. Each tool is sent to the model as a name, a description, and a JSON Schema of its arguments. That list travels inside every request, so the tool count sets two costs at once: input tokens (money and context-window space) and selection difficulty (the chance the model calls the wrong tool). The benchmark below separates those two effects instead of guessing at them.

Data flow you are about to build:

questions.jsonl ──► bench.py ──► MCP server (env TOOL_COUNT=N)
                        │                 │
                        │           tools/list → name + description + inputSchema
                        ▼                 ▼
              usage.prompt_tokens + chosen tool name ──► results.csv

Checklist before you start:

python3 -m venv .venv && source .venv/bin/activate
pip install --upgrade pip
pip install mcp openai
python -c "import mcp, openai; print('ok')"

Expected output: ok. If the import fails, the package did not install into the active virtualenv — re-run with .venv/bin/pip install mcp openai.

Step 1. Build a server whose tool count is a variable

What to do: create server.py that reads TOOL_COUNT from the environment and registers exactly that many tools, and inspect_tools.py that starts it over stdio and prints how many tools came back plus the byte size of their definitions.

Why: a fixed server cannot be swept from 5 to 100 tools. Making the count an environment variable means one code path, one process, five configurations — so a difference in the results is caused by the count, not by different code.

How to check: inspect_tools.py must print one line per configuration with tools= equal to the requested N.

# server.py
import os
from mcp.server.fastmcp import FastMCP

TOOL_COUNT = int(os.environ.get("TOOL_COUNT", "5"))
mcp = FastMCP("tool-count-bench")

def build(i: int):
    def handler(query: str) -> str:
        return f"shard-{i:03d}: no records for '{query}'"
    handler.__name__ = f"search_shard_{i:03d}"
    handler.__doc__ = (
        f"Search documentation shard {i:03d}. "
        f"Call this only when the question names shard {i:03d}."
    )
    return handler

for i in range(TOOL_COUNT):
    mcp.tool()(build(i))

if __name__ == "__main__":
    mcp.run(transport="stdio")   # stdio = JSON-RPC over stdin/stdout
# inspect_tools.py
import asyncio, json, os, sys, time
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client

async def inspect(n: int):
    params = StdioServerParameters(
        command=sys.executable,
        args=["server.py"],
        env={**os.environ, "TOOL_COUNT": str(n)},
    )
    async with stdio_client(params) as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()
            t0 = time.perf_counter()
            tools = (await session.list_tools()).tools
            dt = (time.perf_counter() - t0) * 1000
            payload = json.dumps([t.model_dump(exclude_none=True) for t in tools])
            print(f"n={n:>3} tools={len(tools):>3} bytes={len(payload):>6} list_ms={dt:.1f}")

async def main():
    for n in (5, 10, 25, 50, 100):
        await inspect(n)

asyncio.run(main())
python inspect_tools.py

Example of the output format (your byte counts will differ — these are placeholders, not measured results):

n=  5 tools=  5 bytes=  1900 list_ms=  40.0
n= 10 tools= 10 bytes=  3800 list_ms=  42.0
n=100 tools=100 bytes= 38000 list_ms=  90.0

Why it works: TOOL_COUNT is read at import time, so each subprocess registers a different number of handlers, and tools/list returns the live registry — the byte size grows because every extra tool adds its name, description, and argument schema.

Step 2. Measure tokens, latency, and correct tool choice at each N

Illustration: Step 2. Measure tokens, latency, and correct tool choice at each N
Illustration: Step 2. Measure tokens, latency, and correct tool choice at each N

What to do: generate a labelled question set, convert the MCP tool definitions into the OpenAI tool format, call the model once per question with tool_choice="auto", and log the chosen tool name plus usage.prompt_tokens.

Why: byte size is a proxy. The number that hits your invoice and your context window is prompt_tokens from the response, and the number that matters for reliability is how often the model picks the intended tool. Measuring both at the same N tells you whether a failure is a budget problem or a selection problem.

How to check: results.csv gains one row per (N, metric) and every accuracy value is between 0 and 100.

# bench.py
import asyncio, csv, json, os, sys
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
from openai import OpenAI

client = OpenAI(base_url=os.environ["LLM_BASE_URL"], api_key=os.environ["LLM_API_KEY"])
MODEL = os.environ["LLM_MODEL"]

async def fetch_tools(n: int):
    params = StdioServerParameters(command=sys.executable, args=["server.py"],
                                   env={**os.environ, "TOOL_COUNT": str(n)})
    async with stdio_client(params) as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()
            return [t.model_dump(exclude_none=True) for t in (await session.list_tools()).tools]

def to_openai(tools):
    return [{"type": "function",
             "function": {"name": t["name"],
                          "description": t.get("description", ""),
                          "parameters": t["inputSchema"]}} for t in tools]

def pick(tools, question: str):
    resp = client.chat.completions.create(
        model=MODEL, tools=tools, tool_choice="auto", temperature=0,
        messages=[{"role": "user", "content": question}],
    )
    calls = resp.choices[0].message.tool_calls or []
    return (calls[0].function.name if calls else None), resp.usage.prompt_tokens

async def run(n: int, rows: list):
    tools = to_openai(await fetch_tools(n))
    available = {t["function"]["name"] for t in tools}
    prompt_tokens, correct, total = 0, 0, 0
    for name in sorted(available):
        question = f"What does {name.replace('search_shard_', 'shard ')} contain?"
        chosen, used = pick(tools, question)
        prompt_tokens, total = used, total + 1
        correct += int(chosen == name)
    rows.append({"tools": n, "prompt_tokens": prompt_tokens,
                 "correct": correct, "total": total,
                 "accuracy_pct": round(100 * correct / total, 1)})
    print(rows[-1])

async def main():
    rows: list = []
    for n in (5, 10, 25, 50, 100):
        await run(n, rows)
    with open("results.csv", "w", newline="") as f:
        w = csv.DictWriter(f, fieldnames=rows[0].keys()); w.writeheader(); w.writerows(rows)

asyncio.run(main())
export LLM_BASE_URL="https://api.example.com/v1"
export LLM_API_KEY="<your-key>"
export LLM_MODEL="<your-model-id>"
python bench.py

<your-model-id> must be replaced with an ID that your provider actually serves; <your-key> with the key from your account. Why it works: prompt_tokens is counted by the server after the tools array is serialised, so it includes the tool definitions — that is the real overhead of running N tools. Because each question names exactly one shard, the expected tool is unambiguous and the accuracy figure is a fair score.

Step 3. Read the table, pick a count, and verify the choice

Illustration: Step 3. Read the table, pick a count, and verify the choice
Illustration: Step 3. Read the table, pick a count, and verify the choice

What to do: fill in the comparison table from results.csv, find the largest N that still meets your accuracy target, then re-run that single N and confirm the numbers reproduce.

Why: a single run can be noise. Reproducing the chosen configuration once more is what turns a number into a decision — and it is the number you will quote when someone asks why the server is split.

How to check: the second run of the chosen N lands within a few percentage points of the first, and results.csv has five rows.

ToolsDefinition bytesInput tokens (prompt)tools/list latency, msCorrect tool choice, %
5from `inspect_tools.py`from `results.csv`from `inspect_tools.py`from `results.csv`
10…………
25…………
50…………
100…………
column -s, -t results.csv
TOOL_COUNT=25 python inspect_tools.py   # reproduce the chosen point

Expected: the table prints aligned columns, and the second inspect_tools.py line shows tools=25 with a byte count close to the value already in your table.

Decision rules, applied to your own numbers:

1. If accuracy is flat from N to the next step but tokens jump, stop at the lower N — the extra tools buy nothing.

2. If accuracy falls at N but tokens are still small, the problem is description quality, not count: make each description state *when* to call the tool, as in Step 1.

3. If tokens approach the context window before accuracy drops, split the server by domain (for example search-* in one server, write-* in another) and connect only the relevant one per session.

4. Record the chosen N in the repository README next to the model ID — the ceiling is model-specific, so it must be re-measured when the model changes.

Troubleshooting

Problem. The client crashes while parsing the first message from the server.

sed -i 's/print(/print(file=sys.stderr, /' server.py   # or delete the prints
TOOL_COUNT=5 python server.py 1>/dev/null   # nothing should appear on the terminal

Problem. The run dies at 50 or 100 tools.

TOOL_COUNT=50 python bench.py   # find the largest N that fits before splitting

Problem. The sweep stops after the first configuration.

Summary and next steps

You now have a reproducible sweep that turns “how many tools should the MCP server expose” into a table of tokens, latency, and accuracy for your own model. Next, cut the tools that never win a question, then re-run the same harness to confirm accuracy held while prompt_tokens fell.

FAQ

What is an MCP server in one sentence?

A separate process that publishes tools — name, description, and argument schema — which an LLM client can list and call over a standard protocol, so the model reaches your code without custom glue per integration.

How many tools should one MCP server expose?

There is no universal number: the ceiling depends on the model's context window and on how distinguishable your descriptions are. Run the sweep above and take the largest N that still meets your accuracy target — the answer is a measurement, not a constant.

Do tool descriptions really cost money?

Yes. They are serialised into the request, so they are billed as input tokens; the prompt_tokens field in results.csv is that cost measured directly. Shortening a description across 100 tools removes that text from every single request.

Can I keep 100 tools and still get good selection?

Sometimes, if each description states precisely when to call it and the schemas stay small. If accuracy still collapses at 100, split by domain or expose a single router tool that dispatches to the rest — then re-measure with the same harness rather than assuming the fix worked.

🤖 AI summary

Related guides

GPT-6.1 Sol Pricing and Access: Plans and Payment Methods

Рассказываем о стоимости новой модели GPT-6.1 Sol, доступных тарифах и легальных способах оплаты из разных стран.

How much RAM and VRAM a local LLM needs: a size and quantization table

Таблица требований к памяти для локальных моделей от 7B до 125B по квантизациям Q4/Q5/Q8 и советы по выбору железа.

AI-Release: Why Tech Moves Faster Than the Media Reports — and How to Always Stay Ahead

Как AI-Release находит фичи Windows, Android, iOS и ИИ-моделей задолго до того, как они появятся в крупных СМИ. Ноль вод