# Auditing an AI agent in 120 seconds: 32 checks and an A–F grade

> AI Release · @ai_release1 · https://ai-release.org/guides/audit-ii-agenta-za-120-sekund-ifixai_en.html

## TL;DR

- Time to complete: ~15 minutes for the first audit; the scan itself is the 120-second run advertised for iFixAi.
- Difficulty: Intermediate (you need a deployed agent and an API key).
- You will need: iFixAi access (web console or CLI — confirm the exact entry point in the docs), an HTTPS endpoint of the agent under test, a read-only API key for that agent, `curl`, `jq`, and a writable folder for the report.

## Requirements and setup
![Illustration: Requirements and setup](https://ai-release.org/guides/img/audit-ii-agenta-za-120-sekund-ifixai-en-1.jpg?v=1793867998)


- OS: Linux, macOS, or Windows with WSL2; `curl` and `jq` present in `PATH`.
- If your iFixAi access is a Python package: an installed Python 3.x and a virtual environment. Package name and minimum Python version — check the documentation.
- A running agent with a reachable HTTPS endpoint. If you have only a chat UI so far, start with [How to turn a regular LLM chat into a full-fledged AI agent](https://ai-release.org/guides/kak-prevratit-obychnyy-chat-s-llm-v-polnotsennogo-ai-ag.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide) — the audit needs an endpoint, not a browser tab.
- An API key for the agent under test, stored in an environment variable, never in the command history.
- Read access to the agent's config (system prompt, tool list, model name). An audit you cannot tie to a version is not reproducible.
- Disk space for the JSON report plus any PDF export; the exact report size — check the documentation.
- Staging, not production. The audit sends real requests to the agent.

Pre-flight checklist — do not start the scan until every box is ticked:

- [ ] Agent endpoint answers with HTTP 200
- [ ] API key exported in the current shell
- [ ] Agent config committed or copied to a snapshot folder
- [ ] Test prompts contain no real customer data
- [ ] Output folder exists and is writable
- [ ] Nobody is editing the system prompt during the run

```bash
curl --version | head -n 1
jq --version
test -n "$AGENT_API_KEY" && echo "key present" || echo "key missing"
```

Why this works: the first two lines prove the tooling exists, the third proves the key reached the shell — all three failures below are caught here before you spend the 120 seconds.

**Example:** `jq --version` prints `jq-1.7.1`; if it prints `command not found`, install `jq` first.

## Step 1. Freeze the agent under test

**What to do:** record the exact version of the agent, snapshot its config, and confirm the endpoint answers before the audit starts.

```bash
export AGENT_ENDPOINT="https://agent-staging.example.com"
export AGENT_API_KEY="$(cat ~/.secrets/agent-staging.key)"

RUN_DIR="audit/$(date +%F)"
mkdir -p "$RUN_DIR"

curl -sS -o /dev/null -w "%{http_code}\n" \
  -H "Authorization: Bearer $AGENT_API_KEY" \
  "$AGENT_ENDPOINT/health"

cp agent.config.yaml "$RUN_DIR/agent.config.yaml"
git rev-parse --short HEAD > "$RUN_DIR/commit.txt"
```

**Why:** an A–F grade is meaningless without a baseline. When you fix a finding and re-run, the only thing that must change between the two reports is the agent, not the endpoint or the prompt.

**How to check:** `curl` prints `200`; `ls "$RUN_DIR"` shows `agent.config.yaml` and `commit.txt`.

**Example:** if your agent's prompt and tools live in a folder tree, mirror that layout in `RUN_DIR` — a naming convention for those folders is described in [ICM in practice: a folder structure that keeps agents manageable](https://ai-release.org/guides/icm-praktika-struktura-papok-agenta.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide).

## Step 2. Run the 32-check audit
![Illustration: Step 2. Run the 32-check audit](https://ai-release.org/guides/img/audit-ii-agenta-za-120-sekund-ifixai-en-2.jpg?v=1793868008)


**What to do:** start the scan against the frozen endpoint, hand it the credentials, and let the full check set — 32 checks — run. The scan window is 120 seconds; if your run goes past it, the run is not finished, not "slow but fine".

The scan needs exactly four inputs, whether you use the console or the CLI:

| Input | Value in this guide | Note |
|---|---|---|
| Target | `$AGENT_ENDPOINT` | the frozen staging URL from Step 1 |
| Credential | `$AGENT_API_KEY` | read-only key, passed by reference to the env var |
| Check set | all 32 checks | partial sets give a grade you cannot compare |
| Output | `audit/<date>/report.json` | one file per run, never overwrite |

```bash
# Template. Command name and flag spelling differ between iFixAi versions —
# take the exact string from the documentation, keep the four inputs above.
<ifixai-command> scan \
  --target "$AGENT_ENDPOINT" \
  --api-key-env AGENT_API_KEY \
  --checks all \
  --output "$RUN_DIR/report.json"
```

**Why:** running the full 32 checks in one pass is what makes the grade comparable across runs; a partial scan moves the letter without moving the agent.

**How to check:** the report file exists and is not empty:

```bash
test -s "$RUN_DIR/report.json" && echo "report written"
jq 'keys' "$RUN_DIR/report.json"
```

**Example:** `jq 'keys'` lists the top-level fields of your report — for instance the grade and the array of individual checks. Use exactly the names it prints in Step 3; field names are version-specific, so check the documentation rather than copying names from another team's script.

While the scan runs, do not touch the system prompt, the tool list, or the model setting. A report that mixes two configs cannot be reproduced.

## Step 3. Read the grade, triage findings, verify with a re-run
![Illustration: Step 3. Read the grade, triage findings, verify with a re-run](https://ai-release.org/guides/img/audit-ii-agenta-za-120-sekund-ifixai-en-3.jpg?v=1793868015)


**What to do:** read the A–F grade, pull out every check that did not pass, fix in order of severity, then re-run the same command against the same endpoint.

```bash
# Adjust the field names to the ones printed by `jq 'keys'` above.
jq -r '.checks[] | select(.status != "pass") | [.severity, .id, .title] | @tsv' \
  "$RUN_DIR/report.json" | sort
```

Then re-run Step 2 into a new folder and compare:

```bash
ls audit/
jq 'keys' audit/2026-01-15/report.json
jq 'keys' audit/2026-01-22/report.json
```

**Why:** the grade is a summary; the fix list is the work. Comparing two reports from the same endpoint is the only proof that a finding is actually closed.

Grade bands and what they mean for your next action — the exact score cut-offs are printed in your report legend, check the documentation:

| Grade | Reading | Action |
|---|---|---|
| A | No blocking findings | Ship, keep the report as a baseline |
| B | Minor findings | Fix in the current sprint, re-run |
| C | Several findings | Fix before release, re-run required |
| D | Serious findings | Block release, fix, re-run |
| F | Critical findings | Stop, treat as an incident |

**How to check:** the number of non-passing checks drops between the two reports, and the grade moves up or stays. If the count is identical, the fix did not reach the running agent — redeploy, then re-run.

**Example:** first run returns grade `D` with findings in tool permissions; you restrict the tool list, redeploy, re-run, and the same check now reports `pass` and the grade moves to `B`. Two JSON files sit side by side in `audit/`.

## Troubleshooting

**Problem. The scan cannot reach the agent endpoint.**
- Ошибка: `curl: (6) Could not resolve host: agent-staging.example.com`
- Причина: wrong hostname, missing VPN, or a typo in `AGENT_ENDPOINT`.
- Решение: `echo "$AGENT_ENDPOINT"`, then `dig +short agent-staging.example.com`. Fix the variable and repeat Step 1: `export AGENT_ENDPOINT="https://<your-real-host>"`.

**Problem. The scan starts but every request is rejected.**
- Ошибка: `HTTP 401` from the health check, or a body like `{"error":"invalid_api_key"}`
- Причина: the key was read from a file with a trailing newline, or exported in a different shell than the one running the scan.
- Решение: `printf '%s' "$AGENT_API_KEY" | wc -c` and compare with the key length in your secret manager; re-export without a newline: `export AGENT_API_KEY="$(tr -d '\n' < ~/.secrets/agent-staging.key)"`.

**Problem. The command exits but no report appears.**
- Ошибка: `ModuleNotFoundError: No module named '<package>'` (Python install) or an empty output file — `test -s` prints nothing.
- Причина: the package name differs from the one you guessed, or the output directory did not exist when the scan started.
- Решение: `mkdir -p "$RUN_DIR" && touch "$RUN_DIR/probe" && rm "$RUN_DIR/probe"` to prove the folder is writable, then confirm the exact package name in the documentation and reinstall with `pip install <package>`.

## Summary and next steps

Freeze the agent, run all 32 checks against that frozen endpoint, and read the A–F grade together with the list of non-passing checks rather than the letter alone. Keep every report in its own dated folder so the next run proves the fix — then wire the same command into CI, and read [Designing AI systems: components, diagrams and common pitfalls](https://ai-release.org/guides/kak-proektirovat-ii-sistemu.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide) before you add more tools to the agent.

## FAQ

**How long does an iFixAi audit take?**
The scan itself is the 120-second run; the full procedure above takes about 15 minutes on a first pass because of setup, snapshotting, and reading the report.

**Do I need production credentials for the audit?**
No. Use a read-only key against a staging endpoint. The audit sends real requests to the agent, so production keys add risk without improving the grade.

**What exactly do the 32 checks test?**
The report lists each check by name and ID. The current list changes between versions — check the documentation, and use `jq 'keys'` on your own report to see what your version returned.

**Can I run the audit automatically in CI?**
Yes, if your iFixAi access is a CLI. Confirm the exit-code behaviour in the documentation first, then gate the pipeline on the failing-check count instead of the letter grade.