AN AI DIAGNOSTIC SYSTEM HIT 83% ON COMPLEX CASES—BY ASKING ONE QUESTION AT A TIME.
The system reasoned through 304 difficult cases step by step, but the result is still a benchmark—not patient care.
An AI diagnostic system from Microsoft Research outperformed physicians in two retrospective case benchmarks by gathering information the way a clinician does: one question, test and updated hypothesis at a time. Its 83.2% result on 304 complex cases is striking; the honest next chapter is real-world validation.
WHAT HAPPENED
The researchers introduced MAI-DxO, an orchestrator that lets language-model agents propose diagnoses, choose the next useful question or test, and revise their thinking as evidence arrives. On 304 difficult New England Journal of Medicine cases, it reached 83.2% diagnostic accuracy; physicians using reference materials reached 36.1%. On de-identified MIMIC emergency records, it reached 80.3% versus 46.8%. The work was peer reviewed in Science. These results do not make it a bedside tool: the study used retrospective text cases, and prospective clinical testing, safety evaluation and workflow design are still required. Microsoft plans a real-world triage study with Beth Israel Deaconess Medical Center.
WHY THIS IS HOPESLOP
The encouraging part is not a machine replacing doctors. It is a more disciplined form of medical AI: one that can ask for missing evidence, expose its sequence of decisions and potentially help clinicians spend scarce tests more wisely. The benchmark is unusually strong, while its limitations are unusually clear. That combination—measurable progress with the caveat intact—is exactly the kind of hope worth tracking.
ORIGINAL SOURCEScience / Microsoft AIRead the original report
A DAILY ALLEGATION OF PROGRESS.
One good thing from the internet, every day. Nothing else.