Anthropic and Theorem use Claude to automate software safety proofs

The era of manual code review is facing a significant technological shift as artificial intelligence begins to handle the most rigorous aspects of software safety. This development could fundamentally alter how developers trust automated code generation systems.

The core challenge in software engineering has long been verifying that code works correctly under all possible conditions. Traditional formal verification is extremely expensive and time consuming. For instance, a team of specialists required twenty person-years to formally verify the small operating system kernel seL4. Consequently, this method was mostly reserved for critical systems like Airbus avionics software where errors are costly.

Anthropic has changed this dynamic by collaborating with partner Theorem to use its Claude model for mathematical verification. The Frontier Red Team allowed Claude to take over the proof work, generating approximately 180 million lines of proof code in just a few weeks. This effort covered test programs from 123 open-source libraries within Google's OSS-Fuzz security project.

The process does not involve writing new code from scratch. Instead, Claude verifies existing software by translating crash tests into mathematical problems and analyzing compiled machine code. The model produces extensive proofs that are then checked by an independent verification program called Rocq. Anthropic describes the process as fully autonomous, meaning it requires almost no human reviewers, although the underlying tools and formal foundations were built by humans at Theorem.

During the proof generation process, Claude identified hundreds of bugs that conventional testing had missed. Many of these issues only appeared under very specific conditions, such as unusually large input files or particular sequences of data access. In one notable example, the team found as many security bugs in a widely used PDF reader in one week as are typically reported for that software in an entire year. Claude also caught a logical error that had remained undetected for over a decade.

Even when a complete proof ultimately fails, the attempt often surfaces real bugs, according to Anthropic. This capability suggests that formal verification can serve as a powerful debugging tool, not just a certification method. The model was allowed to fix many of the bugs it found, streamlining the development cycle further.

However, the approach has strict practical limits. Anthropic notes that formally verified does not mean error-free in every context. The proofs hold only under specific conditions, including a particular processor architecture, single-threaded programs, and inputs under 64 MB. In real-world scenarios, these libraries often run on different hardware configurations, which means the proofs are conditional rather than absolute.

The computational cost of this verification is also significant. Individual proofs can consume hundreds of millions of tokens, though Anthropic has not disclosed the total project costs. Despite this, the efficiency is improving. Newer models are becoming more capable, with Opus 5.5 solving half the proofs with about 40 percent fewer tokens than its predecessor, Opus 5.

This project strengthens the argument for broader software automation. If AI can write code and guarantee its correctness under defined conditions, the industry can shift focus from basic functionality to proven safety. This could reduce the time developers spend on manual review and lower overall coding costs.

Looking ahead, Anthropic expects formal verification with AI to become as routine as standard testing. Theorem's roadmap already targets critical infrastructure libraries such as glibc, OpenSSL, and cURL. If successful, this approach could lead to a major leap in software reliability and automation across the tech sector.