It is easy to overinterpret this based on the headline, the doctors were actually at a slight disadvantage. This isn't how they normally work, this is a little more like a med school pop quiz:
An AI and a pair of human doctors were each given the same standard electronic health record to read – typically including vital sign data, demographic information and a few sentences from a nurse about why the patient was there. The AI identified the exact or very close diagnosis in 67% of cases, beating the human doctors, who were right only 50%-55% of the time.... The study only tested humans against AIs looking at patient data that can be communicated via text. The AI’s reading of signals, such as the patient’s level of distress and their visual appearance, were not tested. That means the AI was performing more like a clinician producing a second opinion based on paperwork.
Also, this is not how ER doctors work? They are not trained for this, nor does it reflect their day-to-day performance. If they would work like this, perhaps they would know a bit more about the nurse writing down those notes, and the kinds of things that particular nurse is likely to miss or overemphasize - just as an example.
The article gives a neat example: In one case in the Harvard study, a patient presented with a blood clot to the lungs and worsening symptoms. Human doctors thought the anti-coagulants were failing, but the AI noticed something the humans did not: the patient’s history of lupus meant this might be causing the inflammation of the lungs. The AI was proved correct.
Which is nice and all, but in the presence of a blood clot, I can understand that treating inflammation instead is not the first thing on a doctor's mind, what with blood clots being potentially life threatening and all. It raises the question; was this a real-life case, and what happened to that patient? Since this is a case for which the correct diagnosis is known, it was eventually correctly diagnosed - presumably then the patient did not die of a blood clot, nor of an uncontrollable fever.
Also, how representative is a patient with Lupus? According to House, MD, it's never Lupus.