At this week’s spring seminar series at the Stanford Cyber Policy Center, Jonathan Chen, MD, PhD delivered a highly entertaining talk on the role of AI in the medical field, full of magic tricks to illustrate AI’s illusion of intelligence. 

Dr. Chen discussed several studies demonstrating that recent large language models often outperform not only human doctors, but also human-AI medical collaborations (the supposed pinnacle of performance). Even with respect to empathy, a seemingly human-specific trait, chatbots tend to outperform their human doctor counterparts; a result that becomes less surprising considering that human doctors simply do not have enough time to meet the overwhelming demands on the medical system, let alone deliver optimally empathetic messages. 

Despite their impressive outputs, however, these language models are still prone to confabulation or ‘hallucination,’ with real consequences. For example, when asking a language model to provide references explaining how opioids improve mortality in heart failure, its response will often seem convincing and credible despite being factually incorrect. A human user therefore must already have the necessary knowledge to verify those veracity of outputs. With 20% of ChatGPT queries being health-related, users need to know that language models are what Dr. Chen calls “auto-complete on steroids,” not arbiters of medical truth.

Dr. Chen posed the broad question: What makes a doctor a doctor? (Or any professional a professional?) What is the point of a medical degree when a chatbot can easily pass the knowledge-based medical tests? Traditional answers might have included qualities like knowledge, empathy, and liability, but as Dr. Chen suggests that value now lies in how we leverage those skills. Chen also highlighted that technological advancements with these language models outpace the peer review process. To stay relevant and valuable, doctors will need to learn how to work with AI tools—viewing them not as oracles, but a means to synthesize information and communicate with their patients in optimal ways. 

NEXT WEEK:

Tuesday, May 6 at 1pm PT as Katy DeCelles, who holds the Secretary of State Professorship of Organizational Effectiveness at the Rotman School of Management at the University of Toronto and is the VMware Women’s Leadership Lab Fellow at the Center for Advanced Study in the Behavioral Sciences (CASBS) at Stanford, gives her talk Scale Dichotomization Reduces Customer Racial Discrimination and Income Inequality. To learn about this and future Cyber Policy Center events, visit CPC's event page.