Sixteen Claude agents built a C compiler: what the experiment measured

In February 2026, Anthropic researcher Nicholas Carlini described an experiment in which 16 Claude Opus 4.6 instances worked on a Rust-based C compiler. The reported result was substantial, but the coordination was not the hierarchy described in this article's earlier version: there was no orchestration agent assigning every task.

Carlini's account and Benj Edwards' Ars Technica coverage instead describe peers working through a shared repository and task locks, supported by a deliberately engineered test environment. The public code is an inspectable artifact, not proof that every generated program is correct. UnliveNews has not reproduced the experiment or spent API funds testing it.

The reported scope

The dated account reports nearly 2,000 Claude Code sessions and about $20,000 in API costs for a roughly 100,000-line compiler capable of building Linux 6.9 on x86, ARM and RISC-V. Ars also reports compilation of projects including PostgreSQL, SQLite, Redis, FFmpeg and QEMU, a 99-percent result on the GCC torture suite and a Doom demonstration.

Those outcomes need their scopes attached. A test-suite percentage describes the suite used, not the probability that an arbitrary future program will compile correctly. A kernel build and a working demonstration establish particular accomplishments; they are not a complete language-conformance or production-reliability certificate.

The cost also has a boundary. It is reported API expenditure, not the total cost of human research, compute infrastructure, model training and the existing tools and tests. Dividing the approximate $20,000 by approximately 2,000 sessions gives an illustrative $10 per session average. That arithmetic is not a forecast of another project's price or a standard session rate.

How peers coordinated without a director

Carlini describes isolated Docker environments, local clones of a shared Git repository and task claims stored as text files. Agents pulled and merged others' work, pushed changes and released locks. He explicitly says he did not use an orchestration agent.

This is a narrower mechanism than a project manager deciding an architecture and delegating fixed compiler roles. The old article invented that narrative and also described implementation details as unavailable despite the published repository. Both claims have been removed.

The distinction is useful because coordination and correctness are separate problems. A task lock can reduce duplicated work; it does not establish that the claimed task is well chosen or that its implementation preserves unrelated behaviour. Git can combine text changes while a semantic interaction between them still needs testing.

The test environment was part of the experiment

Ars describes Carlini shaping feedback for the agents, including concise test summaries, reduced test sampling for fast iteration and a GCC reference strategy when multiple agents converged on the same kernel problem. The scaffolding supplied an environment in which failures could become tractable tasks.

That human contribution is more precise than saying people manually wrote every fix or that no human work mattered. Separate the written compiler code from the design of its verifier and task environment. Each can be essential even when only one appears in a headline about who wrote the code.

A proposed evaluation record should therefore list the artifact, allowed tools, tests, reference implementation and human interventions. Record the successful output separately from the process used to obtain it. Otherwise, changing a verifier or supplying an oracle can disappear from a supposedly comparable result.

External assistance and limitations still count

The February report identifies a missing 16-bit x86 backend that required GCC assistance for the real-mode boot step, bugs in the project's assembler and linker, weak generated-code efficiency and regressions introduced by later changes. These are limitations of the reported state, not independently refreshed November benchmarks.

The public repository's human-written warning says the code has not been validated for correctness and is not recommended for use. Its documentation also warns that claims in the generated docs may be wrong. A repository's existence enables inspection; it does not remove the need for validation.

When reading a claim that an artifact is dependency-free, ask whether that means its implementation, a particular build path or an entire demonstrated workflow. The older GCC-assisted boot account and a later repository description need their dates and scopes, rather than being flattened into one universal claim.

A concrete comparison for future demonstrations

Imagine two hypothetical compilers passing 90 of 100 tests. One might fail ten rejected-input tests; another might silently produce wrong output on ten accepted programs. The equal pass percentages do not establish equal consequences. Test identity and failure type matter alongside the total.

For this experiment, the useful follow-up questions concern a specified revision and workload: which inputs compile, which outputs behave as expected, which external tools were needed and which cases remain unresolved? Those questions do not require inventing the original agent hierarchy or declaring a single benchmark the end of compiler engineering.

The documented accomplishment is a substantial generated artifact under a particular environment and verifier. Preserving both the results and the assistance needed to obtain them makes the demonstration informative without turning it into a general guarantee of autonomous software correctness.

Sources

Nicholas Carlini's February 5 account: https://www.anthropic.com/research/building-c-compiler

Public compiler repository: https://github.com/anthropics/claudes-c-compiler

Original coverage: https://arstechnica.com/ai/2026/02/sixteen-claude-ai-agents-working-together-created-a-new-c-compiler/