THE APPARENT CONTRADICTION
● The Reconstruction benchmark put frontier models at 3 to 15 percent recovering a paper's ideas from its bibliography alone.
● OpenAI's Astra resolved ten problems open for at least a decade, including one unanswered since 1999, for roughly $2,000 in compute.
● Both are true. The difference is whether the problem was already stated.
What Astra actually did
On 1 August 2026 OpenAI announced that an internal version of Astra — a model family with no release date, no pricing, and a US government security review still to pass — produced new results on ten open problems across mathematics and theoretical computer science.
The named results:
- An explicit construction of a non-sofic group, open since Mikhail Gromov introduced soficity in 1999
- A disproof of Connes's rigidity conjecture on von Neumann algebras
- A proof of Ehrhart's volume conjecture
- New bounds on high-dimensional sphere-packing density
- Three problems from the Erdős catalogue, including problem 183 on multicolour Ramsey numbers
Total compute cost across all ten: roughly $2,000 at GPT-5.6 Sol API rates.
The part that makes it different from every previous claim
YOU DO NOT HAVE TO TRUST OPENAI
OpenAI published a 249-page manuscript, a 62-page methodology narrative, and Lean 4 proof certificates for all ten results on GitHub under Apache 2.0.
Lean is a proof assistant. Its kernel checks every step against axioms and returns a binary answer — the proof compiles or it does not. The repository reports a "sorry" count of zero, meaning no step anywhere is left unproven. Anyone with the Lean compiler can verify all ten without a mathematics PhD and without taking OpenAI's word for anything.
That is genuinely new. Every AI capability announcement of the past three years has shared one structural weakness: the company making the claim is the only party able to evaluate it. Benchmarks get contaminated, demos get curated, and the argument becomes about whether the number means anything.
Compare the alternative. The Erdős unit distance result in May 2026 required nine external mathematicians to read and co-sign the argument. That is strong validation, but it is social — it depends on qualified people being available and willing, and it cannot be reproduced without them. A Lean certificate replaces that with a compiler.
Thomas Bloom, who curates the Erdős problems database, called the August results big news and placed them above the May unit-distance work. Bloom is a useful reference point here — he is the same person who publicly called an earlier OpenAI claim a dramatic distortion in 2025, when a model was said to have solved ten Erdős problems and had in fact located existing papers. That post was later deleted.
So how does this square with 3 to 15 percent?
The Reconstruction benchmark, published this month, hands a model a research paper's bibliography and nothing else — no full text, no author information, no post-publication signal. The task is to recover the paper's core ideas. Frontier models scored 3 to 15 percent. A four-model Swiss-tournament pipeline reached 42 percent.
The two results measure different things, and the distinction is precise:
|
Astra on open problems |
Reconstruction benchmark |
| Was the question stated? |
Yes — famous, precisely formulated, decades old |
No — the model must work out what to ask |
| Success criterion |
A compiler says yes or no |
Judged similarity to what a researcher chose |
| Search space |
Enormous, but with a definite target |
Unbounded, no target |
| What it demonstrates |
Execution against a hard specified goal |
Choosing which goal is worth pursuing |
Nobody had to decide that constructing a non-sofic group was worth doing. Gromov posed it in 1999 and mathematicians have wanted it since. Astra's achievement was finding a path to an answer everyone already agreed was valuable — an extraordinarily hard search, and still a search with a destination.
Reconstruction removes the destination. That is where the 3 to 15 percent lives.
THE SAME SHAPE, THREE DOMAINS
Software: Linear's telemetry showed coding agents took teams from 21 weekly pull requests to 65 — and total development time rose. Agents write code faster; humans still decide what to build and whether it is right.
Biology: Claude designed protein binders hitting 14 of 15 lab-verified targets at double the industry rate. The targets were chosen by people.
Mathematics: Astra resolved ten open problems for $2,000. All ten had been posed by humans, some decades earlier.
Three domains, three weeks, one pattern. Steep on execution. Flat on choosing the target.
What the critics are right about
Three caveats, all fair:
A Lean build proves the theorem as formally stated inside Lean. It does not confirm the formal statement captures the open problem as mathematicians understood it. Translating an informal conjecture into formal syntax is itself a judgement, and that judgement is not machine-checked. This is the substantive objection and it has not been resolved.
None of the ten has been peer reviewed. Verification and review are different processes answering different questions.
Nobody outside OpenAI can use Astra. No release date, no pricing, a security review pending. The results are checkable; the capability is not reproducible by anyone else. OpenAI researcher Noam Brown added his own perspective on scale: no Millennium Prize Problems, yet.
What this means if you are not a mathematician
| If you are... |
The useful read |
| Using AI for hard problems |
Specify the target precisely. That single act is what moves a task from the 3 percent column to the 100 percent one |
| Evaluating capability claims |
Ask whether anyone outside the vendor can check it. Astra is the first major claim where the answer is yes |
| Worried about research jobs |
The part being automated is finding the path. Choosing the destination is not |
| Building anything verifiable |
Where a checker exists — compilers, test suites, formal proofs — models are far more useful than where judgement decides |
| Reading the next AI-does-science headline |
The question is who posed the problem. It is almost always a person |
The genuinely interesting implication
The transferable lesson from Astra is not about mathematics. It is that the useful frontier for AI is anywhere a cheap automatic checker exists.
Lean is a checker for proofs. A test suite is a checker for code. A wet lab is a checker for protein binders. In each of those domains, a model can generate at volume and the checker discards what is wrong — which is exactly why $2,000 of compute is enough to resolve a question open since 1999.
Where no cheap checker exists — is this the right research direction, is this the right product to build, is this argument persuasive — models perform much closer to 3 to 15 percent. That is not a training gap. It is the absence of a verification loop.
If you want more from these tools, the highest-leverage thing you can do is not a better prompt. It is building a way to automatically check the answer.
FAQ
Did OpenAI's Astra really solve ten open math problems?
It produced results on ten problems open for at least a decade, with Lean 4 certificates published on GitHub under Apache 2.0. The certificates are machine-verifiable and the repository reports zero unproven steps. None has been peer reviewed.
Can I verify the proofs myself?
Yes. That is the point. With the Lean compiler installed you can run all ten certificates. The kernel returns a binary result, so no mathematics expertise is needed to check that the proofs compile.
What is the catch?
A Lean build confirms the proof is valid for the theorem as formally stated in Lean. Whether that formal statement faithfully captures the original open problem is a human judgement, and it is not machine-checked.
Can I use Astra?
No. It is unreleased, with no pricing and no date, and OpenAI says it must pass a US government security review before any public rollout.
Does this contradict the finding that models score 3 to 15 percent on generating ideas?
No. Astra worked on problems already posed and precisely stated. The Reconstruction benchmark removes the problem statement and asks the model to work out what question is worth asking. Different tasks, different results.
Why did it only cost $2,000?
Because a checker made the search cheap. When a compiler can instantly reject wrong answers, a model can generate enormous numbers of candidates and keep only what verifies. That economics does not hold where correctness is a matter of judgement.