THE SHORT ANSWER
● Vendor harness — 99.9% - OpenAI's tools-enabled configuration, nearly $19,000 per run
● Neutral harness — 62.7% - ARC Prize's standard harness, the one every other model uses
● Gap — More than 37 points on an identical model
● Still true — 62.7% is the top score on ARC-AGI-3
OpenAI published 99.9% for GPT-6 Astra on ARC-AGI-3 when it launched the model on 3 September 2026. Nvidia chief executive Jensen Huang cited that number three days later to say AGI had arrived. ARC Prize has now run the same model through its own standard harness and recorded 62.7%, calling the higher figure not valid.
What separates the two numbers
| OpenAI tools-enabled | ARC Prize standard |
|---|
| Score | 99.9% | 62.7% |
| Built by | The model vendor | The benchmark owner |
| Applied to competitors | No | Yes, all of them |
| Reported run cost | Nearly $19,000 | Not stated |
| Comparable to other models | No | Yes |
| Lower-compute variants | Not reported | 59.3% and 54.8% |
| Counts as AGI, per ARC Prize | Claimed | No |
Neither number is fabricated
This is the part worth being precise about. OpenAI did not invent 99.9%; it measured a real configuration. The objection is not dishonesty, it is incomparability. A harness that gives a model tools, retries and a $19,000 compute budget, applied to one entrant and not the others, produces a number that cannot sit on a leaderboard next to anything.
The neutral run also shows the compute sensitivity directly: lower-compute Astra variants scored 59.3% and 54.8%. So between 54.8% and 99.9% there is a continuum of real measurements of the same model, and the only one that means anything in comparison is the one produced under the same conditions as everyone else.
Why this is not an isolated dispute
Two things happened in the same week. OpenAI reportedly revised five published benchmark figures after its launch post. And a Shandong University study found that harness choice alone moves coding-agent cost by up to 3x with no consistent accuracy gain, with a median 13% of hard tasks flipping pass-to-fail purely from switching harness. Read together, those say the harness is not a detail of the measurement - on some benchmarks it is most of it.
What to do with a published benchmark number
Three questions, in order. Who built the harness? Is it the same harness the comparison models ran under? And what did the run cost? If the answers are vendor, no, and unstated, the number is marketing rather than measurement - however accurately it was obtained.
On Astra specifically, the headline that survives scrutiny is still a good one: the best score anyone has recorded on ARC-AGI-3, by a clear margin, under neutral conditions. ARC Prize's position is simply that this is not AGI.