Microsoft AI System Outperforms Generalists on Complex Diagnostics

For patients with complex medical conditions, a new diagnostic system from Microsoft offers a potential second opinion that may outperform individual generalist physicians. The system, named MAI-DxO, correctly identified diagnoses in difficult simulated cases at an 83% rate, more than double the 36% accuracy of primary care doctors and general internists working individually. This performance gap suggests that AI coordination could address the growing mismatch between medical demand and available expertise.

The technology operates by coordinating multiple AI agents that function like a panel of specialists. These agents question the patient, order necessary tests, and debate the final diagnosis. According to a study published in Science on Thursday, this collaborative approach allows the system to handle complex cases that are typically reserved for physician teams or specialists, rather than general practitioners.

Dominic King, vice president of Health at Microsoft AI, described the findings as substantial. He noted that the primary challenge in current healthcare delivery is the imbalance between the demand for medical expertise and the supply of that expertise. By publishing these results, Microsoft joins Google and OpenAI in asserting that its AI capabilities can match or exceed those of human doctors in specific diagnostic contexts.

The study’s data indicates that much of the performance improvement stems from using more powerful underlying AI models, with the MAI-DxO orchestration layer adding incremental gains. Microsoft tested the system using models from various providers, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The highest results were achieved using OpenAI’s GPT-5 and GPT-5.5 models, though the benefits of the orchestration system varied by model strength.

Cost and accuracy were both measured in the evaluation. When using GPT-5 with a basic prompt, the system achieved 78.9% accuracy at an estimated cost of $8,400 per case. Adding the MAI-DxO layer reduced costs by approximately $3,000 but only increased accuracy by about four points. However, with the newer GPT-5.5 model, the orchestration layer raised accuracy from 72% to nearly 87% at a slightly lower cost. These gains were less evident in emergency department cases, where accuracy and costs remained similar regardless of whether the orchestration layer was used.

The system proved most beneficial for complex cases and when paired with less capable models. For instance, Gemini 2.5 Flash improved from 46.4% accuracy on its own to nearly 69% with MAI-DxO. Microsoft researchers noted that accuracy gains naturally diminish as models approach theoretical maximums, but the orchestration helps optimize both accuracy and cost. A spokesperson stated that the system addresses the broader challenge of balancing these two factors.

Mustafa Suleyman, CEO of Microsoft AI, said the company is collaborating with the Mayo Clinic to train a healthcare-specific foundation model, with release planned for next year. He believes this foundation will enable significant leaps in medical AI. Suleyman predicted that by 2027, the company could achieve a breakthrough in streaming highly accurate predictions about how medical records will unfold, providing clinicians with valuable foresight.

To validate the technology in real-world settings, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston. The study will evaluate the clinical reasoning of MAI-DxO across different hospital environments, including urgent care and emergency departments. Patients will receive standard treatment while consenting to parallel engagement with the AI, allowing researchers to compare AI recommendations against human physician decisions. King described this as the largest study of its kind, aiming to provide compelling evidence for the wider deployment of such systems.