# How to Speed Up Python Code with an AI Agent: Profiling and Rewriting the Hot 5%

> AI Release · @ai_releaseorg · https://ai-release.org/guides/kak-uskorit-python-kod-s-pomoschyu-ii-agenta-profilirov_en.html

## TL;DR
- Time to complete: 45 minutes
- Difficulty: Intermediate
- You will need: Python 3 with the standard library (`cProfile`, `pstats`, `timeit`) — run `python3 -V` to confirm; a git repository with a clean working tree; one slow script or function plus a way to run it; a coding AI agent (CLI or chat) that can read the files you paste or point it at.

## What you will get
- `perf/profile.out` and a saved `perf/before.txt` report listing the functions that eat your runtime, sorted by `tottime` (time spent inside the function body itself).
- `perf/hotspots.md` — a self-contained briefing packet (source lines, call counts, benchmark command, behaviour constraints) that you hand to the agent, so it never guesses what to optimise.
- `perf/after.txt`, a reviewed `git diff` limited to the hot file, and a `test_hot.py` assertion script that proves the rewrite returns the same values before you merge.

## Quick start
```bash
git checkout -b perf/hot-slice
python3 -m cProfile -o profile.out bench.py
python3 -c "import pstats; pstats.Stats('profile.out').strip_dirs().sort_stats('tottime').print_stats(10)"
python3 -m timeit -s "import bench" "bench.score_rows(bench.ROWS)"
```
Details and checks are in the steps below.

## Prompt to copy
```prompt
You are a Python performance engineer. Work only on the file and function named below. Do not touch any other file.

Target: the function score_rows in bench.py, called 1 time from the __main__ block.
Benchmark command that must be used for verification: python3 -m timeit -s "import bench" "bench.score_rows(bench.ROWS)"
Correctness command: python3 test_hot.py

Hard constraints:
- Keep the function signature and the return type unchanged.
- Keep the output order identical to the current implementation. Do not sort, deduplicate or reorder results.
- Behaviour must stay identical for: empty input, one row, several rows for the same user, and rows with equal scores.
- Use only the Python standard library. Add no new dependencies.
- Do not modify bench.py, test_hot.py, or the number of rows in ROWS. If you believe the benchmark is wrong, say so instead of editing it.
- Prefer algorithmic fixes (fewer passes, a dict instead of a nested scan) over micro-optimisations such as local variable caching.

Here is the profiler summary for the function (pstats, sorted by tottime):
PASTE_TOP_LINES_HERE

Here is the current source of the function, exactly as it exists on disk:
PASTE_FUNCTION_SOURCE_HERE

Deliver, in this order:
1. A unified diff (git diff format) limited to the target function.
2. One short paragraph: what changed, and the complexity before and after.
3. The exact command I run to verify, and the exact output that means "no regression".
Do not claim any speed-up you have not measured. If you cannot measure it, write "not measured".
```

Replace before sending: `PASTE_TOP_LINES_HERE` with the 5–10 `pstats` lines for the target function, `PASTE_FUNCTION_SOURCE_HERE` with the real source, and the function/file names if your target differs.

## Requirements and setup
![Illustration: Requirements and setup](https://ai-release.org/guides/img/kak-uskorit-python-kod-s-pomoschyu-ii-agenta-profilirov-en-1.jpg?v=1795337786)

- Linux, macOS, or Windows with a shell; commands below are written for bash.
- Python 3 installed: `python3 -V` must print a version. Any version supported by python.org works — check the documentation for the current release notes.
- `git` installed and the project committed, so `git diff` shows only the agent's change.
- A benchmark entry point (`bench.py`) with fixed data, so two runs are comparable.
- A coding agent with access to your repository or the ability to accept pasted files. If your agent currently lives in a plain chat window, see [How to turn a regular LLM chat into a full-fledged AI agent](https://ai-release.org/guides/kak-prevratit-obychnyy-chat-s-llm-v-polnotsennogo-ai-ag.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide) for the minimal setup (file reading, command execution).
- Free disk space: a `cProfile` dump is small, but keep a few hundred MB free for virtualenvs and caches.

## Step 1. Lock a baseline and capture the profile
**What to do:** write a deterministic benchmark, commit it, then profile it with `cProfile` (the standard-library profiler that records how many times each function ran and how long it took).

```python
# bench.py
import random

random.seed(0)
ROWS = [
    {"user": f"u{random.randint(1, 5000)}", "score": random.randint(0, 100)}
    for _ in range(2000)
]

def score_rows(rows):
    totals = []
    for row in rows:
        total = 0
        for other in rows:
            if other["user"] == row["user"]:
                total += other["score"]
        totals.append(total)
    return totals

if __name__ == "__main__":
    print(sum(score_rows(ROWS)))
```

```bash
git checkout -b perf/hot-slice
git add bench.py && git commit -m "add benchmark for hot path"
python3 -m cProfile -o profile.out bench.py
mkdir -p perf
python3 -c "import pstats; pstats.Stats('profile.out').strip_dirs().sort_stats('tottime').print_stats(10)" | tee perf/before.txt
```

**Why:** the agent can only improve what you can measure. `random.seed(0)` makes every run use the same 2000 rows, so a "before" and "after" number are comparable; `-o profile.out` writes stats to a file instead of dumping them to the terminal, which is what the next command reads.

**How to check:** `ls -l profile.out` shows a non-empty file, and `perf/before.txt` starts with lines shaped like this (your numbers will differ):

```
         <N> function calls (<N> primitive calls) in <T> seconds

   Ordered by: tottime
   List reduced from <A> to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
```

Your target function should appear in that top-10 list. If it does not, raise the `10` to `30` and look again.

**Example:** the columns mean `ncalls` = how often it was called, `tottime` = seconds spent in the function body only, `cumtime` = seconds including everything it calls, and `filename:lineno(function)` = where to look. In the sample above, `score_rows` is called once but its `tottime` dominates — that is your hot slice.

## Step 2. Build the hot-slice packet and hand it to the agent
![Illustration: Step 2. Build the hot-slice packet and hand it to the agent](https://ai-release.org/guides/img/kak-uskorit-python-kod-s-pomoschyu-ii-agenta-profilirov-en-2.jpg?v=1795337791)

**What to do:** take only the top entries from the profile and write a briefing file that contains everything the agent needs: source, numbers, the benchmark command, and the constraints.

| `pstats` column | What it tells you | Use it to decide |
|---|---|---|
| `ncalls` | how many times the function was invoked | high `ncalls` + high `tottime` = per-call overhead or a quadratic call pattern |
| `tottime` | time inside the body, callees excluded | this is the primary target for a body rewrite |
| `cumtime` | time including callees | use it to walk down into the child that actually burns the time |
| `percall` | `tottime / ncalls` | very small `percall` with huge `ncalls` means call-count reduction wins |

```bash
# line numbers come from the filename:lineno(function) column
sed -n '8,16p' bench.py > perf/snippet.py
wc -l perf/snippet.py
head -5 perf/before.txt > perf/hotspots.md
cat perf/snippet.py >> perf/hotspots.md
printf '\nConstraints: same signature, same output order, stdlib only, output order must stay identical for every user.\nBenchmark: python3 -m timeit -s "import bench" "bench.score_rows(bench.ROWS)"\n' >> perf/hotspots.md
```

**Why:** a profile line alone ("`score_rows` took the most time") gives the agent nothing to verify against, and it will happily invent a benchmark. Pasting the real source, the real command, and the ordering constraint keeps the rewrite inside your test harness. Keeping this material in a small folder also keeps the agent's instructions next to the code it touches — the same discipline described in [ICM in practice: a folder structure that keeps agents manageable](https://ai-release.org/guides/icm-praktika-struktura-papok-agenta.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide).

**How to check:** run this checklist on `perf/hotspots.md` before pasting it anywhere.

- [ ] the function source is complete (no `...` and no truncation)
- [ ] the `ncalls` / `tottime` line for the function is included
- [ ] the exact benchmark command is written out
- [ ] the output-order constraint is stated explicitly
- [ ] the correctness command (`python3 test_hot.py`) is named

Then send the prompt from the "Prompt to copy" section, with the two paste slots filled.

**Example:** for the `bench.py` above, the packet contains `score_rows` with its single `ncalls` line, the nested loop source, and the instruction "output order must stay identical" — because the obvious "fix" (returning a dict of totals) changes the return type and breaks every caller.

## Step 3. Apply, verify, and keep the change
![Illustration: Step 3. Apply, verify, and keep the change](https://ai-release.org/guides/img/kak-uskorit-python-kod-s-pomoschyu-ii-agenta-profilirov-en-3.jpg?v=1795337795)

**What to do:** apply the agent's diff on the branch, prove correctness with a fixture test, then measure again with the same command.

```python
# test_hot.py
import bench

def test_fixture():
    rows = [
        {"user": "a", "score": 1},
        {"user": "a", "score": 2},
        {"user": "b", "score": 5},
    ]
    assert bench.score_rows(rows) == [3, 3, 5], bench.score_rows(rows)
    assert bench.score_rows([]) == []
    print("ok")

test_fixture()
```

```bash
python3 test_hot.py
git diff --stat
python3 -m timeit -s "import bench" "bench.score_rows(bench.ROWS)"
python3 -m cProfile -o profile_after.out bench.py
python3 -c "import pstats; pstats.Stats('profile_after.out').strip_dirs().sort_stats('tottime').print_stats(10)" | tee perf/after.txt
```

**Why:** the fixture asserts the exact expected list `[3, 3, 5]`, so a rewrite that changes ordering, drops duplicates, or returns a dict fails immediately. `git diff --stat` proves the agent stayed inside the target file. Comparing `perf/before.txt` with `perf/after.txt` shows whether the function moved out of the top of the list — the real success criterion, not one lucky timer run.

**How to check:** `python3 test_hot.py` prints `ok`; `git diff --stat` lists only `bench.py` (one file changed); `perf/after.txt` either no longer shows `score_rows` in the top-10 body-time list, or shows it with a smaller `tottime`. If the test prints `AssertionError`, the rewrite changed behaviour — reject it.

**Example:** after the fix, the inner `for other in rows` scan is replaced by one pass that builds a per-user total in a dict and then maps rows back to it in the original order, so `score_rows` drops out of the top-10 `tottime` list while `test_hot.py` still prints `ok`.

## Troubleshooting

**Problem. `pstats` refuses your sort key.**
- Error: `ValueError: Invalid sort key: 'totaltime'`
- Cause: `pstats` accepts only a fixed set of keys — `ncalls`, `tottime`, `cumtime`, `calls`, `time`, `name`, `filename`, `line`, `nfl`. `totaltime` does not exist.
- Solution: use the real key: `python3 -c "import pstats; pstats.Stats('profile.out').strip_dirs().sort_stats('tottime').print_stats(10)"`.

**Problem. The analysis command cannot find the stats file.**
- Error: `OSError: [Errno 2] No such file or directory: 'profile.out'`
- Cause: you ran `python3 -m cProfile bench.py` without `-o profile.out`, so the report went to stdout and no file was created; or you ran the `pstats` command from a different directory.
- Solution: rerun with `-o`, from the repository root, then confirm: `python3 -m cProfile -o profile.out bench.py && ls -l profile.out`.

**Problem. The rewrite passes the benchmark but breaks callers.**
- Error: `AssertionError: [5, 3, 3]` inside `test_hot.py`
- Cause: the agent replaced the nested scan with a dict, then returned the dict's values — that is, it reordered results by user instead of preserving input order.
- Solution: reject the diff and restate the constraint in the prompt (`Keep the output order identical to the current implementation`). A correct version keeps one pass over `rows` to build `totals_by_user`, then a second pass that emits `totals_by_user[row["user"]]` in input order.

## Summary and next steps
You now have a reproducible benchmark, a profile that names the few functions eating the runtime, and a reviewed patch that keeps behaviour intact. Next, repeat Step 1 on the next entry in `perf/before.txt`, and add `python3 test_hot.py` to your CI so later optimisations cannot silently change results.

## FAQ

**How much faster will the rewrite be?**
There is no fixed number — it depends entirely on your code and data. The only trustworthy figure is the one you measure with the same `timeit` command before and after; if you cannot measure it, do not report it.

**Can I skip the benchmark and just let the agent optimise?**
No. Without a fixed `bench.py` and a fixture test, you cannot tell a real speed-up from a lucky run, and you cannot tell whether the output still matches.

**Does `cProfile` distort the numbers?**
Yes, profiling adds overhead per call, so profiler numbers are for finding hot spots, not for reporting speed-ups. Use `timeit` without the profiler for the before/after figure, and only compare profiler runs with other profiler runs.

**What if the hot function is inside a third-party library?**
You cannot patch it safely, so change what you control: reduce how often you call it, cache its results, or batch the calls. Exact options differ per library — check the documentation for that package before editing anything.

**Which model or agent should I use?**
Any coding agent that can read the source files and run commands works with this workflow. Model names, versions, and prices change constantly — check the documentation of the tool you already have rather than trusting a fixed recommendation.
