The figure in circulation is 83% against 36%: Microsoft's MAI-DxO diagnosing complex New England Journal of Medicine cases against generalist physicians, published in Science and covered everywhere this week. The question nobody is asking is how much of that belongs to the orchestration layer and how much to the model underneath it. The underlying paper answers it.
In the ablation published with Sequential Diagnosis with Language Models, OpenAI's o3 running a plain baseline prompt scored 78.6% at an average $7,850 per case. The same model inside MAI-DxO with no budget constraint scored 81.9% at $4,735. That is the orchestrator's marginal contribution on the strongest base model tested: 3.3 accuracy points, against a 40% reduction in cost. Budgeted, it scored 79.9% at about $2,396 - 1.3 points of accuracy for a 69% cost cut. The ensemble configuration reached 85.5% at $7,184, which is 6.9 points for no real saving.
So the orchestrator is a cost-control mechanism that also improves accuracy slightly, which is a genuinely useful thing to have built. It is not the source of the gap with physicians. In the same paper 21 physicians averaged 19.9% at $2,963 per case across a 56-case set, taking 11.8 minutes, 6.6 questions and 7.2 tests each. The distance between 19.9% and 78.6% is the foundation model. The distance between 78.6% and 81.9% is Microsoft.
One more number complicates the marketing. The paper states that MAI-DxO raised accuracy for all sufficiently capable models by roughly 11 percentage points on average - but the strongest model tested gained 3.3. An 11-point average with a 3.3-point result at the top means the average is carried by the weaker models in the set; GPT-4o alone managed 49.3%. That points the opposite way from the authors' claim that improvements in model capability do not necessarily diminish the benefit of multiple agents. On this evidence the better the base model, the less the scaffolding adds.
Two caveats in both directions. These ablation figures come from the arXiv preprint built on o3 and GPT-4o, and the Science version reports updated headline numbers - 83% against 36% with references and 22% without, and 80% against a 47% average on emergency cases - whose per-model breakdown is not in the main tables. And the newest models are absent entirely: no GPT-6 Astra, no Gemini 4 Argon, no Opus 5.5. Clinical testing at Beth Israel Deaconess is due within two months, which is where the claim gets its first real test.
Sources
arXiv preprint · TechCrunch · ExplainX