Microsoft AI Beats Doctors On Hard Cases

Microsoft is preparing to test its AI diagnostic system in a major real-world trial at Boston’s Beth Israel Deaconess Medical Center, following new results published Thursday in Science that claim its technology outperformed human generalists on complex cases. The study, released on October 22, marks a significant escalation in the race to define the future of clinical decision support. Microsoft AI CEO Mustafa Suleyman described the findings as a critical step toward achieving what he calls “medical superintelligence,” a state where artificial intelligence surpasses human-led health systems in reasoning and coordination.

The system at the center of this development is known as MAI-DxO, or DxO. It functions as a digital panel of doctors, coordinating multiple AI agents to question patients, order tests, and debate potential diagnoses. In the simulation, which utilized complex cases from the New England Journal of Medicine, DxO was correct 83% of the time. By comparison, primary care doctors and general internists working individually with reference materials were correct only 36% of the time. While these cases are typically handled by specialists or teams in real practice, the gap between the AI and individual generalists was stark.

Dominic King, vice president of Health at Microsoft AI, highlighted the urgency of the issue, noting a growing mismatch between the demand for and supply of medical expertise. “There is something very substantial and exciting here,” King said. By publishing these results, Microsoft joins Google and OpenAI in asserting that its AI capabilities can match or exceed those of human physicians. However, the data suggests that much of the performance improvement stems from the underlying power of the models themselves, with Microsoft’s orchestration system providing an additional layer of efficiency.

Cost and accuracy were both measured in the study. When using OpenAI’s GPT-5 model on its own with a basic prompt, the system scored 78.9% on complex cases at an estimated cost of $8,400 per case. DxO reduced that cost by approximately $3,000 while raising accuracy by about four points. The benefits were less pronounced for emergency department cases, where GPT-5 performed similarly with and without the DxO assist. With the newer GPT-5.5 model, released in April 2026, DxO boosted accuracy on complex cases from 72% to nearly 87% at a slightly lower cost, though again, emergency case results remained flat at around 85%.

The orchestration layer proved most valuable when paired with less capable models. For instance, Gemini 2.5 Flash scored 46.4% on its own but jumped to nearly 69% when assisted by DxO. Researchers noted that accuracy gains naturally diminish as models approach theoretical limits. A Microsoft spokesperson explained that DxO helps optimize both accuracy and cost, addressing a broader challenge in healthcare delivery. Suleyman emphasized that the company is collaborating with the Mayo Clinic to train a healthcare foundation model, slated for release next year.

Suleyman, who co-founded DeepMind before joining Microsoft in 2024, believes these efforts will lead to significant breakthroughs. He predicted that by 2027, the company will be on the verge of a “coding style breakthrough” for medical records, allowing for highly accurate predictions of how a patient’s condition might unfold. He views this foresight as invaluable for clinicians. “We’re just true believers,” Suleyman said, expecting healthcare models to improve by an order of magnitude within the next two to three years as they enter a production flywheel of real-world iteration.

The upcoming trial with Beth Israel Deaconess will evaluate DxO’s clinical reasoning across various hospital settings. Patients will receive standard care, including triage and specialist consultation, while those who consent will also interact with the AI system. Microsoft and its partners will then compare the AI’s recommendations against those of human doctors. King described the plan as potentially the largest study of its kind, aiming to provide compelling evidence for scaling these systems beyond emergency settings into urgent care environments.

This initiative coincides with a broader manifesto from Microsoft on building medical superintelligence, defined as the capacity to surpass contemporary human-led health systems across the full arc of care. The company aims to move beyond simple benchmarking tasks like answering exam questions to integrating directly with clinicians for better outcomes. Microsoft already sells dictation and radiology tools to healthcare providers and views DxO as an expansion of these efforts. Suleyman stated that the company aims to have the “absolute number one state-of-the-art medical clinical decision support products” in the coming years.

The timing of the release is notable, as Google recently published a similar clinical study with Beth Israel in The Lancet, evaluating its AMIE conversational diagnostic AI in real-world urgent care visits. Both major tech firms are now engaging in direct, real-world validation of their diagnostic capabilities. For Microsoft, this represents a strategic differentiator among frontier labs, driven by the goal of accelerating the introduction of advanced AI into the global healthcare system. The focus remains on creating a robust infrastructure that enables real breakthroughs in how medical records are handled and how clinical decisions are made.