New Study Shows ChatGPT Outperforms Real Doctors in Providing Medical Advice on Depression

Health

ChatGPT Outperforms Primary Care Physicians in Depression Treatment, New Study Finds

In a recent study comparing ChatGPT (versions 3.5 and 4) with primary care physicians, AI tools were found to provide better therapeutic recommendations for the treatment of depression. The findings, published in the open access journal Family Medicine and Community Health, emphasize the potential of AI in enhancing decision-making in primary healthcare.

The study involved testing ChatGPT by comparing its responses to 1,249 French primary care doctors, predominantly women. The researchers used hypothetical case studies of patients diagnosed with mild to moderate depression and inputted case vignettes into the AI chatbot’s interface. The researchers then asked ChatGPT, “What do you think a primary care physician should suggest in this situation?”

The results showed that in mild depression cases, primary care doctors recommended referral for psychotherapy in only 4% of cases, while ChatGPT-3.5 and ChatGPT-4 selected this option in 95% and 97.5% of cases, respectively. In severe cases, most doctors recommended psychotherapy and prescribed drugs (44.5%), whereas the AI chatbots proposed this option more frequently (72% for ChatGPT-3.5 and 100% for ChatGPT-4) in line with clinical guidelines. It is worth noting that neither ChatGPT model recommended prescribing drugs exclusively, which four out of 10 doctors suggested.

The researchers, from Israel and the UK, concluded that ChatGPT’s therapeutic proposals align with accepted guidelines for depression treatment, without exhibiting biases related to gender or socioeconomic status. They highlighted the potential of AI tools in improving healthcare decision-making and patient outcomes. However, they also acknowledged the need for further research to address the risks and ethical issues associated with AI technology. They emphasized that ChatGPT should not be considered a substitute for human clinical judgment.

ChatGPT has also demonstrated success in other areas, as reported by Business Insider in June. The AI model passed exams such as the Uniform Bar Exam and the SAT, scoring higher than the average. It also passed the Graduate Record Examinations, although its performance on the writing test was relatively lower.

In conclusion, the study’s findings highlight the effectiveness of ChatGPT in providing therapeutic recommendations for depression treatment. AI tools like ChatGPT have the potential to enhance healthcare decision-making, although further research and consideration of ethical issues are necessary.