But the report published Wednesday also stressed ER doctors needn't hang up their scrubs just yet, with the chatbot potentially able to speed up diagnosis but not replace human medical judgement and experience.
Scientists examined 30 cases treated in an emergency service in the Netherlands in 2022, feeding in anonymised patient history, lab tests and the doctors' own observations to ChatGPT, asking it to provide five possible diagnoses.
They then compared the chatbot's shortlist to the same five diagnoses suggested by ER doctors with access to the same information, then cross-checked with the correct diagnosis in each case.
Doctors had the correct diagnosis in the top five in 87 percent of cases, compared to 97 percent for ChatGPT version 3.5 and 87 percent for version 4.0.
"Simply put, this indicates that ChatGPT was able to suggest medical diagnoses much like a human doctor would," said Hidde ten Berg, from the emergency medicine department at the Netherlands' Jeroen Bosch Hospital.
Co-author Steef Kurstjens told AFP the survey did not indicate that computers could one day be running the ER, but that AI can play a vital role in assisting under-pressure medics.
"The key point is that the chatbot doesn't replace the physician but it can help in providing a diagnosis and it can maybe come up with ideas the doctor hasn't thought of," Kurstjens told AFP.
Large language models such as ChatGPT are not designed as medical devices, he stressed, and there would also be privacy concerns about feeding confidential and sensitive medical data into a chatbot.
- 'Bloopers' -
And as in other fields, ChatGPT showed some limitations.
The chatbot's reasoning was "at times medically implausible or inconsistent, which can lead to misinformation or incorrect diagnosis, with significant implications," the report noted.
The scientists also admitted some shortcomings with the research. The sample size was small, with 30 cases examined. In addition, only relatively simple cases were looked at, with patients presenting a single primary complaint.
It was not clear how well the chatbot would fare with more complex cases. "The efficacy of ChatGPT in providing multiple distinct diagnoses for patients with complex or rare diseases remains unverified."
Sometimes the chatbot did not provide the correct diagnosis in its top five possibilities, Kurstjens explained, notably in the case of an abdominal aneurysm, a potentially life-threatening complication where the aorta artery swells up.
The only consolation for ChatGPT: in that case the doctor got it wrong too.
The report sets out what it calls the medical "bloopers" the chatbot made, for example diagnosing anaemia (low haemoglobin levels in the blood) in a patient with a normal haemoglobin count.
"It's vital to remember that ChatGPT is not a medical device and there are concerns over privacy when using ChatGPT with medical data," concluded ten Berg.
"However, there is potential here for saving time and reducing waiting times in the emergency department. The benefit of using artificial intelligence could be in supporting doctors with less experience, or it could help in spotting rare diseases," he added.
The findings -- published in the medical journal Annals of Emergency Medicine -- will be presented at the European Emergency Medicine Congress (EUSEM) 2023 in Barcelona.
Space Medicine Technology and Systems
|Subscribe Free To Our Daily Newsletters|
Kombucha: Ally for Moon and Mars
Virgin Galactic notches fourth spaceflight in four months
China continues to make strides in space breeding technique
Musk biography describes troubled tycoon driven by demons
New DOD-funded project will develop morphing hypersonic engine
Phase Four's next generation Maxwell Propulsion System Achieves Flight Heritage
Relativity Space signs lease on historic NASA Test Stand
Another Martian Weekend" Sols 3943-3945
Sols 3936-3939: Double the Fun
China publishes new datasets obtained by Mars, lunar probes
NASA's completes Oxygen-Generating Experiment MOXIE
Tianzhou 5 spacecraft burns up on Earth reentry
Crew of Shenzhou XV mission honored for six-month space odyssey
China solicits names for manned lunar exploration vehicles
From rice to quantum gas: China's targets pioneering space research
Vodafone and Amazon's Project Kuiper to extend connectivity in Africa and Europe|
Intuitive Machines announces $20M equity investment
SpaceX sends 22 new Starlink satellites into orbit in 60th launch of 2023
LeoStella and Hera Systems Establish Strategic Alliance
Apple to update iPhone 12 in France over radiation
UK Lords hand govt defeat in homebuilding pollution vote
AI-driven tool makes it easy to personalize 3D-printable models
Scientists pursue drug to protect US troops from radiation sickness
On the road to spotting alien life
Scientists detect and validate the longest-period exoplanet found with TESS
New giant planet evidence of possible planetary collisions
Hot Jupiter blows its top
Possible existence of Earth-like planet predicted in Outskirts of Solar System
SwRI will lead Hubble, Webb observations of Io, Jupiter's volcanic moon
In the service of planetary science, astrophysics and heliophysics
Mysterious Neptune dark spot detected from Earth for the first time
|Subscribe Free To Our Daily Newsletters|