Microsoft Claims AI Diagnostic Tool Tops Doctors on Tough Cases

WHO:

TechCrunch - Mustafa Suleyman is Microsoft’s AI CEO.

Techmeme - Microsoft and Microsoft’s AI CEO Mustafa Suleyman are associated with the Diagnostic Orchestrator.

WHAT:

Gizmodo - Exact test results were 83.2% diagnostic accuracy for the AI versus 36.1% for generalist physicians with references; in emergency department cases, the AI scored 80.3% versus physicians’ 46.8%.

Gizmodo - Additional testing showed improved results using GPT-5.5. Microsoft said the system reached diagnoses at half or one-third the cost, based on estimates from simulated cases rather than actual medical expenses.

Gizmodo - The researchers said they would expect the performance gap to narrow if the system were compared with specialist physicians rather than generalists.

Gizmodo - Researchers described the system’s sequential process: it starts with a chief complaint, then selects follow-up questions and tests before reaching a diagnosis. Its agents propose diagnoses, select tests, challenge assumptions, and weigh costs.

Gizmodo - Researchers said the agent approach may help counter cognitive biases such as anchoring on an initial diagnosis or overlooking alternatives.

Gizmodo - Researchers said prior benchmarks had not evaluated language models’ diagnostic abilities sequentially and proposed the Sequential Diagnosis Benchmark, or SDBench.

Gizmodo - A Harvard University and Beth Israel Deaconess Medical Center study found OpenAI’s o1-preview reasoning model outperformed physicians diagnosing emergency-room patients during triage.

Gizmodo - A 2025 study found that doctors who used AI to identify potential cancer risks during colonoscopies performed worse after the AI was removed.

The Verge - Microsoft’s Diagnostic Orchestrator is powered by OpenAI’s GPT-5; its researchers said it outperformed frontier AI models running independently.

The Verge - The Diagnostic Orchestrator was first developed when GPT-4.1 was the most advanced model available. Researchers said later testing suggests that “improvements in model capability do not necessarily diminish the benefits” of multiple agents working together.

The Verge - Microsoft has tested the tool only on past medical cases, so its performance in real clinical settings remains unclear. Mustafa Suleyman said, “It needs clinical validation.”

The Verge - Microsoft has not announced when or how the technology might be commercialized; its rollout would need to meet medicine’s regulatory and compliance requirements.

The Verge - Suleyman said, “Next year we’re going to see extremely impressive deployments of intelligent systems in production.”

The Verge - Microsoft’s Diagnostic Orchestrator scored about 83% accuracy on cases from hundreds of *The New England Journal of Medicine* reports, compared with roughly 36% for generalist physicians using references and 22% for those without references.

The Verge - On emergency department cases from Beth Israel Deaconess Medical Center’s patient-admissions database, the system scored 80% accuracy versus a 47% average for human physicians, and reached diagnoses at a lower estimated cost.

The Verge - Microsoft compiled the cases into its own benchmark, SDBench, which it plans to open-source.

The Verge - The study’s peer-reviewed paper did not include newer models such as GPT-6 Astra, Meta’s Muse Wonder Full, Google’s Gemini 4 Argon, or Anthropic’s Opus 5.5.

The Verge - Microsoft’s AI health lead Dr. Dominic King coauthored the study and outlined five additional areas for AI in healthcare, including coordinating clinical care, making treatment decisions, and handling the human side of medicine.

TechCrunch - Microsoft’s new AI diagnostic system beat generalist doctors on hard cases, according to a study in *Science*.

TechCrunch - Most of the improvement appeared to come from the underlying AI models rather than Microsoft’s system itself.

Techmeme - Microsoft says its GPT-5-powered Diagnostic Orchestrator diagnoses conditions up to four times more accurately than doctors.

Techmeme - Suleyman told *The Verge*: “We're going to build something that is indistinguishable from human performance ... in the next couple of years.”

WHERE:

The Verge - Microsoft plans to test the system in hospital settings with Beth Israel Deaconess Medical Center.

WHY:

The Verge - Suleyman said AI could remove some of medicine’s “drudgery” and free clinicians to spend more time with patients.

The Verge - Microsoft says it wants to develop “medical superintelligence” capable of outperforming the best human-led healthcare systems.

HOW:

Gizmodo - The study, “Sequential Diagnosis with Language Models,” was first posted as a non-peer-reviewed preprint on arXiv before publication in *Science*.

Techmeme - The Diagnostic Orchestrator uses agents that mimic panels of experts to diagnose conditions.

(UPDATED October 23, 2026 05:05 UTC)

Microsoft’s Diagnostic Orchestrator, an AI system powered by OpenAI’s GPT-5, outperformed generalist doctors on challenging diagnostic cases in a study published in *Science*. It scored 83.2% on cases drawn from medical reports, compared with 36.1% for doctors using references. In emergency department cases, the system reached 80.3% accuracy, against physicians’ 46.8%. The orchestrator works through a case step by step: its AI agents ask follow-up questions, choose tests, weigh costs, and challenge possible assumptions before suggesting a diagnosis. Microsoft says it reached diagnoses at a lower estimated cost, though those figures came from simulated cases, not actual medical bills. Researchers caution that the comparison was with generalists, not specialists, and the system has only been tested on past cases. Microsoft plans hospital testing with Beth Israel Deaconess Medical Center; AI chief Mustafa Suleyman says clinical validation is needed.

(UPDATED October 23, 2026 01:31 UTC)

Microsoft says its GPT-5-powered Diagnostic Orchestrator performed better than generalist doctors on difficult medical cases, though it has not been tested in real clinical care. The system uses multiple AI agents to mimic a panel of specialists. In tests on cases drawn from medical journal reports, it scored about 83 percent accuracy, compared with roughly 36 percent for doctors using references and 22 percent without them. On emergency department cases, it reached 80 percent, versus a 47 percent average for physicians, at a lower estimated cost. Researchers said some gains may come from the underlying AI models, not the orchestration system alone. Microsoft plans hospital testing with Beth Israel Deaconess Medical Center and says its benchmark, SDBench, will be open-sourced. AI CEO Mustafa Suleyman stressed that clinical validation is needed. Microsoft has not said when or how it might commercialize the technology.

(UPDATED October 22, 2026 22:13 UTC)

Microsoft says its AI diagnostic system, the GPT-5-powered Diagnostic Orchestrator, outperformed generalist physicians on difficult medical cases in a study published in *Science*. The system uses AI agents that mimic panels of experts. On cases drawn from hundreds of reports in *The New England Journal of Medicine*, it scored about 83% accuracy, compared with 36% for doctors using references and 22% without them. On emergency cases from Beth Israel Deaconess Medical Center, it reached 80% accuracy, versus 47% for physicians, at a lower estimated cost. Microsoft says most of the improvement came from the underlying AI models, not its orchestration system. The company plans to open-source its SDBench test set and evaluate the tool in hospitals with Beth Israel. Microsoft AI CEO Mustafa Suleyman says it aims to match human performance in the next couple of years.

(UPDATED October 22, 2026 21:11 UTC)

Microsoft says its AI diagnostic system can outperform generalist doctors on difficult medical cases. The GPT-5-powered Diagnostic Orchestrator was reported in a study published in *Science*, and Microsoft says it can diagnose conditions up to four times more accurately than doctors. The system works through AI agents designed to mimic a panel of medical experts, coordinating their assessments to reach a diagnosis. But the study found that most of the improvement appeared to come from the underlying AI models, rather than from Microsoft’s orchestration system itself. Microsoft’s AI CEO, Mustafa Suleyman, told *The Verge* the company aims to build a system that is indistinguishable from human performance “in the next couple of years.” The results raise the profile of AI in medical diagnosis, while highlighting that the models behind the system may account for much of its performance.