🔍 Read the full analysis: Claude And Formal Verification: The Promise And The Boundaries on ThorstenMeyerAI.com
Prime made for students and young adults
- Fast, free delivery for dorm and study essentials
- Prime Video and Amazon Music included
- Member-only deals
TL;DR
Anthropic’s Frontier Red Team and Theorem say Claude produced Rocq-checked proofs for 239 fuzz-test entry points across 123 open-source libraries. The proofs establish a specific no-crash property for compiled RISC-V programs under stated conditions—not that the libraries are bug-free or that other builds are safe.
Anthropic’s Frontier Red Team and collaborator Theorem reported on October 22 that Claude helped prove a defined safety property for 239 fuzz-test entry points in 123 open-source libraries. The machine-checked proofs apply to compiled RISC-V programs and show that, for inputs under 64 MB and under the project’s assumptions, the programs do not crash with an AddressSanitizer memory error or failed assertion; they do not establish that the libraries are free of all bugs.
The team’s method starts with an existing fuzz test rather than a broad specification of what a library should do. Each library and test was compiled into a RISC-V executable and instrumented with AddressSanitizer. A formal model and generated specification described the property to be proved: that the test would not crash for any input below the stated 64 MB limit. Claude generated proofs in Rocq, a proof assistant, and the Rocq checker verified them against the specification.
Anthropic said the work produced proofs totaling about 180 million lines of Rocq over several weeks. A typical proof was about 600,000 lines and addressed a binary with a median length of 18,000 RISC-V instructions. The authors said hundreds of bugs were discovered during the work, including issues they said were unlikely to be reached by ordinary fuzzing. These are the project’s reported results; the source material does not provide an independent evaluation of the full set of findings.
When a proof attempt exposed a counterexample or failure, Claude could propose a code patch and attempt a proof against a regenerated specification. Anthropic said it is publishing proofs that did not require patching, some previously unpublished trusted components, and specifications for fuzz tests it could not prove. The article says some proofs consumed hundreds of millions of tokens, making computational cost a significant constraint, though newer models and reused proofs can reduce the work.
Infographic 01 · From test to proof
Existing fuzz tests became specifications. Claude proved them.
Over a few weeks, Claude proved at the machine-code level that hundreds of real fuzz entry points can no longer crash, after patching the bugs it found.
The pipeline
From an existing test to a checked proof
- Compile. The library and its existing fuzz test become a RISC-V executable, instrumented with AddressSanitizer.
- Specify. Theorem’s RISC-V and Linux process semantics plus the no-crash property mechanically yield the specification.
- Prove. Claude writes the proof in Rocq. A failed attempt points at a possible bug; Claude patches it and proves the new spec.
- Check. The Rocq proof checker verifies the proof. Virtually none of the proof needs human review.
Fuzzing vs. proof
Same entry point, different guarantee
| Fuzzing | Proof | |
|---|---|---|
| Inputs | Fuzzing: Generated samples, typically capped at 1 MB | Proof: Every input under 64 MB |
| Paths | Fuzzing: Guided by code coverage, no guarantee | Proof: Every path reachable from the entry point |
| Result | Fuzzing: Bugs found so far | Proof: No crash left to find on the modelled system |
Typical proof: about 600,000 lines for a binary with a median length of 18,000 RISC-V instructions.
“If Claude can prove this spec, the fuzz test on RISC-V becomes superfluous.”Anthropic, 22 Oct 2026
Source: Anthropic, “LLMs are making formal software verification feasible at a much larger scale” (22 Oct 2026).
Powered by Thorsten Meyer AI
From Fuzzing to Machine-Checked Guarantees
Fuzzing tests a program by trying many inputs, but it cannot by itself show that every possible path has been exercised. A successful proof can cover all inputs and execution paths represented by its model and specification. That makes the approach potentially useful for finding rare errors that random or mutation-based testing does not encounter, especially in code that processes large or unusually structured inputs.
The approach also addresses a longstanding cost problem. Formal verification has often required substantial specialist effort; the source article cites the seL4 microkernel project as taking two person-years to build and 20 person-years to verify. Here, Claude generated large proof artifacts while Rocq checked them mechanically. That division matters: the model’s output is not trusted merely because it sounds convincing, but the result still depends on the correctness and scope of the specification, models and checker.
For developers and users, the practical promise is narrower than a general guarantee of secure software. If extended and made affordable, this workflow could provide machine-checked assurance for specific behaviors in selected components. The reported token demands and the restricted build environment show why the findings do not yet amount to routine verification of the versions most people install.
AddressSanitizer memory error detection tool
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Test Becomes the Specification
Formal verification requires a precise statement of what a program must do. Instead of defining broad correctness for each library, the project used existing fuzz tests as the basis for a narrower claim: that running the test on the compiled program would not trigger a crash. The claim includes the test’s assertions, so some tests can encode behavior beyond memory safety. For example, the source article says an LZO compression test checks that decompressing compressed data returns the original.
The reported proof scope is specific: single-threaded, static RISC-V binaries linked against musl, with inputs under 64 MB and the stated instrumentation and models. It does not automatically apply to x86 or ARM builds linked against glibc, which the source says are more common in practice. Anthropic describes the work as, to its knowledge, the first large-scale use of AI to apply formal verification to production software at assembly-code level; that is the authors’ characterization, not an independently established priority claim.
The proof checker reduces the amount of generated material that people must inspect, but does not remove the trusted base. The source identifies the formal models of RISC-V instructions and Linux processes, the theorem statements and the Rocq checker as components that must be trusted, along with supporting hardware, the operating system and the OCaml compiler on which Rocq relies.
As an affiliate, we earn on qualifying purchases.
Where the Proofs Stop
The results establish only the properties encoded in the specifications for the modeled binaries and inputs. A no-crash proof does not show that outputs are correct in every respect, that access controls always work, or that a program has no exploitable flaw. The source article describes a logic bug involving a file-loading restriction and notes that some bugs do not cause crashes, underscoring the gap between the proved test property and complete security.
It is also not clear from the supplied material how many reported bugs were independently confirmed, fixed, assigned vulnerability identifiers or present in current releases. The proof scope does not cover other architectures and build configurations automatically. Nor does the material provide a complete accounting of token use, financial cost, or how much the newer models and proof reuse reduced those costs.
Some proof attempts did not finish, according to the source article. A failure to complete is not itself proof that a program is unsafe, although the attempt may still reveal a bug or a difficult path. The published specifications and artifacts can help others examine the work, but the source material does not establish how widely those results have been independently reproduced.
formal verification software tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Published Proofs and Broader Testing
Anthropic said it is releasing the successful proofs that required no patch, previously unpublished trusted components, and specifications for fuzz tests it could not prove. Those materials provide a basis for outside researchers to inspect the theorem statements, assumptions and proof artifacts. The supplied report does not give a schedule for additional releases or describe an independent audit.
The next practical test is whether the approach can be made less costly and applied to more real-world build configurations. That would require extending or adapting the models and proofs for other architectures and environments, while keeping the claims tied to what was actually checked. Until then, developers should treat the results as evidence about particular compiled programs and properties—not as certification of every release of the libraries involved.
As an affiliate, we earn on qualifying purchases.
Key Questions
What did Claude prove?
It helped produce Rocq-checked proofs that 239 fuzz-test entry points in compiled RISC-V programs would not trigger an AddressSanitizer memory error or failed assertion for inputs under 64 MB, under the stated assumptions.
Does this mean the libraries are bug-free?
No. The proofs cover specific tests, properties, binary builds and input limits. They do not establish that a library has no other bugs or that every behavior is correct.
Why could formal proofs find bugs fuzzing missed?
Fuzzers try many inputs but may not reach rare paths. A proof attempt must account for all paths represented by its model and specification, which can expose errors triggered only by unusual input sizes or call sequences.
Do the results apply to common x86 and ARM versions?
Not automatically. The reported proofs cover single-threaded static RISC-V binaries linked against musl. The source says they do not directly cover x86 or ARM builds linked against glibc.
What remains to be checked?
Researchers can examine the released proofs, specifications and trusted components. The bug count, fixes, independent reproduction, broader platform coverage and full cost of the work remain unclear from the supplied material.
Source: ThorstenMeyerAI.com
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
