Anthropic's FlashProve: zkSNARK proofs of LLM inference on H100 at 110 tokens/s

⚡ The gist in 5 seconds

• Anthropic has released FlashProve — a system of zkSNARK proofs that a response was generated by the exact model claimed.
• On Llama 4 Maverick (400B, mixture of experts), a single H100 generates proofs at ~110 tokens/s, with proof size ~19 MB per 1,024-token request.
• The code, Lean proofs, and documentation are released under Apache 2.0, but this is an intermediate result, not a finished product.

🔍 What happened

Anthropic published a research post titled "Progress on provable inference for large language models" along with the FlashProve system. The post is credited to Roy Rinberg and Alex Allain from Anthropic, Alessandro Chiesa (EPFL, StarkWare), Miranda Christ (MIT, RESI), Rahul Dalal, Giacomo Fenzi (EPFL), Ben Mayer, and Guy Weissenberg (EPFL). On Hacker News, the post scored 185 points and 87 comments.

The problem the authors tackle is simple to state. You pay for a specific model, but there's no way to verify that the tokens actually came from it. A routing service could swap in a cheaper model to save money, or an attacker could rewrite the token stream in transit. You can't catch this from the response text alone, even if the answer is good. The scheme works like this: the provider "commits" to the model's weights, then proves that each response was generated by exactly those weights. They chose zkSNARK for its hiding and efficiency properties. On Llama 4 Maverick (400B parameters, MoE, 4-bit quantized weights, integer arithmetic on prefill), a single H100 produces proofs at ~110 tokens/s with a proof size of ~19 MB per 1,024-token request. That's ~70–95x the GPU time of the inference itself on vLLM. Prior open-source systems lagged by orders of magnitude. The package includes code (a prover, a fast C++ verifier, and a "provably correct" verifier in Lean), Lean proofs of the high-level claims, a technical report, and a protocol specification written by Claude itself. According to the authors, Claude also improved the zkSNARK schemes for inference.

🧠 Our take

What strikes me here isn't the cryptography but the admission in the text: Claude "is good at optimization but often makes decisions that are hard to evaluate." That's why the high-level safety guarantees were written out in Lean — so a human can verify them formally rather than take them on faith. In our assessment, this matters far more than the flashy numbers: the team is proactively building a cage for its own AI assistant within research.

What does this change? For everyday chat — almost nothing, honestly. Nobody will pay a 70–95x GPU-time overhead for a borscht recipe. But for independent model evaluators, auditors, and enterprise customers going through a routing intermediary — quite possibly. We believe the market here isn't about "trusting Anthropic" but about a bargaining chip in negotiations: "prove it's your model, not a cheap swap." And there's another layer — requirements around "AI pacing," i.e., confirming that the provider is running the version regulators agreed to.

Who wins. Infrastructure providers with their own hardware and large customers who need the paperwork. Who loses — I wouldn't rush to say. Open-source systems will close part of the gap, and FlashProve itself is open under Apache 2.0, so they can use it. The real losers are gray-market routers, whose business depends on swaps going unnoticed.

⚠️ What we don't know

The benchmarks cover a single model and a single configuration — Llama 4 Maverick in 4-bit quantization. The post says nothing about how the scheme performs on dense models of other sizes or on non-standard engines. There's also no comparison with closed industrial systems: "orders of magnitude faster than open ones" refers only to open ones. The Lean proofs cover the high-level claims, not the entire codebase. And crucially: the system hides the weights but not the architecture — this is explicitly stated. Verify the numbers against the technical report, not the retellings in the HN comments.

Prediction: within two quarters, at least one external fork or independent FlashProve benchmark on a different model will appear — provided the technical report contains enough detail for reproduction.