From the source
From the source
Benchmark shows AI orchestrator outperforms physicians on complex diagnostic cases.
Microsoft AI introduced the Sequential Diagnosis Benchmark (SD Bench) based on 304 NEJM case records and the Microsoft AI Diagnostic Orchestrator (MAI-DxO).
MAI-DxO paired with OpenAI's o3 achieved 85.5% diagnostic accuracy, compared to a mean accuracy of 20% among 21 experienced physicians.
The orchestrator is model-agnostic and designed to emulate a virtual panel of clinicians.
From the source
The best performing setup was MAI-DxO paired with OpenAI’s o3, which correctly solved 85.5% of the NEJM benchmark cases. For comparison, we also evaluated 21 practicing physicians from the US and UK, each with 5-20 years of clinical experience. On the same tasks, these experts achieved a mean accuracy of 20% across completed cases.
microsoft.ai
