SUN, OCTOBER 25, 2026
Independent · In‑Depth · Practitioner‑Tested
Any LLM Research

Benchmark Claim Verification Prompts

Five prompts for checking a published AI benchmark score before you repeat it. Written after ARC Prize reran GPT-6 Astra on its own neutral harness and recorded 62.7% against the 99.9% the vendor published, a gap of more than 37 points on an identical model. Paste a launch post or model card and work through these in order.

⌨️ 5 prompts 🕐 Updated Oct 25, 2026
💡 How to use these prompts: Replace everything in [BRACKETS] with your specific details before sending. Click Copy to copy any prompt to your clipboard instantly.
1
Separate the score from the setup
Find out what each number was actually measured under
Read this model announcement. For every benchmark figure, state the exact score, the benchmark name and version, who ran the evaluation, what harness or configuration was used, and the reasoning or compute setting. Put each in a table. Where the page does not state one of those, write not stated rather than inferring it.
2
Test comparability
Separate comparable results from incomparable ones
For each benchmark figure in this announcement, state whether the comparison models were run under the same harness, the same compute budget and the same tool access. List every figure where they were not. Those are the numbers that cannot be placed on a leaderboard.
3
Find the missing comparisons
Notice who was left out of the table
List the competing models that a reader would expect to see in this comparison and that are absent. For each, say whether the page gives a reason. Then say which absence most changes how the results read.
4
Price the run
Tell a reproducible result from a demonstration
From this announcement, work out or locate what it cost to produce each headline figure: compute budget, number of attempts allowed, tool calls permitted, and any best-of-n or ensemble configuration. State which figures depended on a setting a normal user could not afford or access.
5
Rewrite the claim honestly
Get a version you can quote without a correction later
Restate this announcement in plain language, keeping only claims the page itself supports under conditions comparable to other models. Mark every figure you dropped and say why. Keep it under 200 words.