# Step-by-step guide: using LLMs for formal software verification and logic bug hunting

> AI Release · @ai_release1 · https://ai-release.org/guides/poshagovoe-rukovodstvo-kak-ispolzovat-llm-dlya-formalno_en.html

## TL;DR

- Time to complete: 45–60 minutes
- Difficulty level: Intermediate
- You will need: a machine with Linux or macOS (Windows with WSL2 also works), Python 3.10+ (run `python3 --version` to check), an API key for any LLM chat/API you already have access to (exact setup — check your provider's documentation), and a small C or Python project you want to check for logic errors.

This guide shows a repeatable workflow: prepare code, ask an LLM to find logical flaws, then confirm each finding with a deterministic tool (compiler warnings, assertions, a test run). The LLM proposes; the verifier confirms.

## Requirements and setup
![Illustration: Requirements and setup](https://ai-release.org/guides/img/poshagovoe-rukovodstvo-kak-ispolzovat-llm-dlya-formalno-en-1.jpg?v=1792789811)


- [ ] OS: Linux (Ubuntu/Debian), macOS, or Windows 11 with WSL2. For installing AI apps on other platforms, see [How to Download and Install AI Apps on Android via RuStore: Step-by-Step Guide](https://ai-release.org/guides/kak-skachivat-i-ustanavlivat-ii-prilozheniya-na-android.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide) if you work from a phone.
- [ ] Python 3.10 or newer: `python3 --version` must print `Python 3.10` or higher.
- [ ] Access to an LLM via web chat or API. Exact model names, limits and pricing change often — check your provider's documentation before starting.
- [ ] A compiler if you verify C code: `sudo apt install build-essential` (Ubuntu) or `xcode-select --install` (macOS).
- [ ] A working directory with your code, e.g. `~/verify-demo`.
- [ ] At least 100 MB of free disk space.

## Step 1. Prepare the code and a baseline check

**What to do:** Create a demo file with an intentional logic bug, then run your compiler/linter to get a baseline of what deterministic tools already catch.

```bash
mkdir -p ~/verify-demo && cd ~/verify-demo
cat > discount.c <<'EOF'
#include <stdio.h>
#include <stdbool.h>

int final_price(int price, bool is_vip, int coupon_pct) {
    int discount = 0;
    if (is_vip) discount += 15;
    discount += coupon_pct;          // bug: discounts can stack over 100%
    return price - price * discount / 100;
}

int main(void) {
    printf("%d\n", final_price(1000, true, 90)); // expect a sane price
    return 0;
}
EOF
gcc -Wall -Wextra -fsanitize=undefined,address -o discount discount.c && ./discount
```

**Why:** The LLM step later is only useful if you know what the machine already found. Compiler warnings and sanitizers (built-in runtime checks for undefined behavior) are deterministic; anything they miss is the target for the LLM pass. This compiles cleanly — the bug (VIP 15% + coupon 90% = 105% discount, so the output is a negative price) is purely logical, which is exactly the class of errors LLMs are good at surfacing.

**How to check:** The command prints a number. If it prints `-50`, you reproduced the bug: a price of 1000 became negative. `gcc` prints no warnings — that silence is your baseline.

**Example:** expected output:

```text
-50
```

## Step 2. Run the LLM analysis pass (the key process)
![Illustration: Step 2. Run the LLM analysis pass (the key process)](https://ai-release.org/guides/img/poshagovoe-rukovodstvo-kak-ispolzovat-llm-dlya-formalno-en-2.jpg?v=1792789819)


**What to do:** Send the code to the LLM with a structured prompt that forces it to enumerate logic errors with line references and severity, instead of writing a vague review.

Save this as `prompt.txt`:

```text
You are a formal-verification assistant. Analyze the code below.
For EACH issue output exactly:
- ID (L1, L2, ...)
- line number
- class: overflow / boundary / off-by-one / invariant violation / unreachable / other
- severity: high / medium / low
- one-line explanation of the violated property
Do not comment on style. If a function is correct, say "no issues".
Code:
<paste discount.c here>
```

Then either paste it into your LLM's web chat, or, if you use an API, call it from Python (endpoint, model name and auth header depend on your provider — check the provider's documentation):

```python
import os, sys, urllib.request

code = open("discount.c").read()
prompt = open("prompt.txt").read().replace("<paste discount.c here>", code)
body = os.environ["LLM_REQUEST_JSON_TEMPLATE"].replace("{PROMPT}", prompt.replace("\\", "\\\\").replace('"', '\\"'))
req = urllib.request.Request(
    os.environ["LLM_API_URL"],               # e.g. https://<your-provider>/v1/... — see provider docs
    data=body.encode(),
    headers={"Content-Type": "application/json",
             "Authorization": "Bearer " + os.environ["LLM_API_KEY"]},
)
print(urllib.request.urlopen(req).read().decode())
```

**Why:** The structured output format (ID + line + class + property) is what makes the next step possible: each finding becomes a checkable claim. A free-form "this code looks risky" answer cannot be verified. Formal verification means proving a property (e.g., "output price is always between 0 and input price"); the LLM proposes candidate violated properties, and you confirm them.

**How to check:** The model's answer contains at least one finding of class `boundary` or `invariant violation` pointing at the `discount += coupon_pct;` line, with severity `high`. If the answer is prose without IDs and line numbers, re-send the prompt — the format instruction was ignored.

**Example:** a correct answer looks like:

```text
L1 | line 7 | invariant violation | high | discount can exceed 100, so final_price returns a negative value
```

If you also want the LLM to fix the code, ask in a second message: "Rewrite `final_price` so the total discount is capped at 100 and the result is never negative." Keep the fix as a separate diff — never let the model silently rewrite the whole file. For a broader look at wiring an LLM into your own services, see [How to add AI to your home server: self-hosted services guide](https://ai-release.org/guides/kak-podklyuchit-ii-k-domashnemu-serveru-poshagovoe-ruko.html?utm_source=tg&utm_medium=channel&utm_campaign=guide_inline&utm_content=guide_to_guide).

## Step 3. Verify each LLM finding and confirm the fix
![Illustration: Step 3. Verify each LLM finding and confirm the fix](https://ai-release.org/guides/img/poshagovoe-rukovodstvo-kak-ispolzovat-llm-dlya-formalno-en-3.jpg?v=1792789828)


**What to do:** Turn each LLM finding into an executable check. For finding L1, add an assertion that encodes the violated property, rebuild, and run:

```bash
cat > check.c <<'EOF'
#include <assert.h>
#include <stdbool.h>

int final_price_fixed(int price, bool is_vip, int coupon_pct) {
    int discount = (is_vip ? 15 : 0) + coupon_pct;
    if (discount > 100) discount = 100;
    int result = price - price * discount / 100;
    assert(result >= 0 && result <= price);   // the property the LLM claimed was violated
    return result;
}

int main(void) {
    assert(final_price_fixed(1000, true, 90) == 100);  // 100% discount cap
    assert(final_price_fixed(1000, false, 0) == 1000);
    assert(final_price_fixed(0, true, 0) == 0);
    return 0;
}
EOF
gcc -Wall -Wextra -o check check.c && ./check && echo "ALL PROPERTIES HOLD"
```

**Why:** This is the core rule of the whole workflow: an LLM finding is a hypothesis until an assertion, a test, or a sanitizer run confirms it. The `assert` line is the formal property written in code; if the fixed function ever returns a negative price, the program aborts with the exact line number.

**How to check:** The last line of output is `ALL PROPERTIES HOLD`, and `./check` exits with status 0 (`echo $?` prints `0`). If an assertion fires, you will see `check: check.c:9: final_price_fixed: Assertion 'result >= 0 && result <= price' failed.` — that means the property is still violated and the fix is wrong.

**Example:** full expected output:

```text
ALL PROPERTIES HOLD
```

Repeat this loop (LLM proposes → you encode the property → assertion/test confirms) for every finding. Two or three iterations usually clear a small file.

## Troubleshooting

**Проблема. The LLM answer is generic prose with no IDs or line numbers**
- Error (symptom): the reply reads like "The code has some potential issues with discount handling…" with no `L1 | line 7 | …` lines.
- Cause: the model ignored the output format section, usually because the code was appended after a long conversation or the format block was truncated.
- Solution: start a fresh conversation and put the format rules *after* the code, or re-send with the phrase "Output must start with the line 'L1 |' or 'no issues'." Verify by checking the first line of the reply.

**Проблема. The LLM reports a bug that does not exist (false positive)**
- Error (symptom): finding says "integer overflow in `price * discount`", but your inputs are small and `./check` passes.
- Cause: the model reasons about possible inputs, not your actual input domain.
- Solution: constrain the claim — reply with "inputs are guaranteed: 0 <= price <= 100000, 0 <= coupon_pct <= 100" and ask it to re-check. Then re-run `./check`; keep only findings that survive the assertion run.

**Проблема. `gcc: error: unrecognized command line option '-fsanitize=undefined,address'`**
- Error: `gcc: error: unrecognized command line option '-fsanitize=undefined,address'`
- Cause: an older compiler or a non-GCC/Clang toolchain without sanitizer support.
- Solution: drop the sanitizer flags and rely on assertions only: `gcc -Wall -Wextra -o discount discount.c && ./discount`. Sanitizer availability depends on your toolchain — check your compiler's documentation.

## Summary and next steps

The workflow is a loop: prepare a baseline with deterministic tools, have the LLM enumerate candidate logic errors as structured claims, then convert each claim into an assertion or test that either holds or fails. Next, scale it by running the same prompt over every file in your repository and keeping a `check_*.c` file per verified property.

## FAQ

**Can an LLM replace a real formal verifier (proof tools)?**
No. In this workflow the LLM only proposes candidate properties and bugs; the proof obligation is discharged by assertions, tests and sanitizers. Treat model output as unverified hypotheses.

**Which model should I use?**
Any LLM you already have access to will follow the structured prompt. Model names, versions and pricing change frequently — check your provider's documentation for the current options.

**Does this work for languages other than C?**
Yes. The loop is language-agnostic: use your language's assertion mechanism (`assert` in Python, `panic`/tests in Go, etc.) and its compiler/linter for the baseline step.

**How do I know a finding is a true positive?**
Only by an executable check: if the assertion or test you wrote for that property fails on the original code and passes after the fix, the finding was real.