NEJM AI

Expert feedback lays the foundation of rigorous research. However, the rapid growth of scholarly production challenges the conventional scientific feedback mechanisms. High-quality peer reviews are increasingly difficult to obtain.    Weixin Liang, MS, et al. created an automated pipeline using GPT-4 to provide comments on scientific papers. The authors evaluated the quality of GPT-4’s feedback through two large-scale studies.     In the authors’ prospective user study, more than half of the users found GPT-4–generated feedback helpful/very helpful, and 82.4% found it more beneficial than feedback from at least some human reviewers. They also identify several limitations of large language model (LLM)–generated feedback.    Through both retrospective and prospective evaluation, the authors find substantial overlap between LLM and human feedback as well as positive user perceptions regarding the usefulness of LLM feedback.     Although human expert review should continue to be the foundation of the scientific process, LLM feedback could benefit researchers, especially when timely expert feedback is not available and in earlier stages of manuscript preparation.     Read the Original Article “Can Large Language Models Provide Useful Feedback on Research Papers? A Large-Scale Empirical Analysis” by Weixin Liang, MS, et al.: https://nejm.ai/3y10ccm    #ClinicalTrials 

The regulation of clinical #ArtificialIntelligence (AI) poses novel challenges for policy makers worldwide.     Existing approaches to assuring the safety and efficacy of AI technologies may suffice for older forms of AI that preceded the development of generative artificial intelligence (GAI). However, the regulation of clinical GAI may require the development of new regulatory paradigms.     An article by David Blumenthal, MD, MPP, and Bakul Patel, MSEE, MBA, reviews approaches in the United States to regulating pregenerative clinical AI and examines a novel possible approach to GAI regulation. The sooner policy makers in the United States and elsewhere tackle the challenges of regulating clinical AI, the sooner its benefits will be made available to people and patients with acceptable assurances of safety and efficacy.    Read the Policy Corner article “The Regulation of Clinical Artificial Intelligence” by David Blumenthal, MD, MPP, and Bakul Patel, MSEE, MBA: https://nejm.ai/4cU7vBG    #AIinMedicine

Medical artificial intelligence (MAI) has evolved from traditional machine learning to deep learning and from supervised methodologies to unsupervised learning paradigms.     Recently, the focus has shifted from task-specific to generalized medical artificial intelligence (GMAI) models. These new #ArtificialIntelligence (AI) models and algorithms still need to be translated to clinical use in various settings.     A Perspective by Weizhi Ma, PhD, et al. discusses the foreseeable transition from specialized MAI models toward more universally applicable models. The authors introduce two concepts as new paradigms: universal medical artificial intelligence (UMAI) and universal health artificial intelligence (UHAI).     UMAI models will be distinguished from GMAI by their capability to emulate critical aspects of human intelligence necessary in clinical practice, particularly physician empathy and intuition.     UHAI further expands beyond addressing disease states, a domain of UMAI, and covers health maintenance and disease prevention, shifting from relying solely on traditional clinical data to integrating broader nonclinical data to allow for the incorporation of AI into a more holistic understanding of human health and disease origin.     Outlined here are key research priorities and future pathways from GMAI to UMAI and subsequently, UHAI, allowing AI to be more integrated, intuitive, and attuned to the needs of patients, physicians, and society.    Read the Perspective “Evolution of Future Medical AI Models — From Task-Specific, Disease-Centric to Universal Health” by Weizhi Ma, PhD, et al.: https://nejm.ai/4cF9Q3C    #AIinMedicine 

Drs. Rohaid Ali and Fatima N. Mirza explain how #AI is revolutionizing patient consent forms. Using ChatGPT, they simplified complex medical language, making information more accessible. Listen to the latest episode of the AI Grand Rounds podcast hosted by NEJM AI Deputy Editors Arjun Manrai, PhD, and Andrew Beam, PhD: https://nejm.ai/ep20 #healthcare #medicine

Volume 1, No. 8 is now available! Here are the latest articles available in the August issue of NEJM AI:    Save this post to revisit later (click the 💬 button at top right of post).    📊 Editorial: Do We Need Data Standards in the Era of Large Language Models? https://nejm.ai/4dcpyDb    🔬 Perspective: A Call for Artificial Intelligence Implementation Science Centers to Evaluate Clinical Effectiveness https://nejm.ai/4f03LjU    ⚕ Perspective: Evolution of Future Medical AI Models — From Task-Specific, Disease-Centric to Universal Health https://nejm.ai/4cF9Q3C    🔍 Original Article: Can Large Language Models Provide Useful Feedback on Research Papers? A Large-Scale Empirical Analysis https://nejm.ai/3y10ccm    ⚖️ Policy Corner: The Regulation of Clinical Artificial Intelligence https://nejm.ai/4cU7vBG    🤖 Case Study: FHIR-GPT Enhances Health Interoperability with Large Language Models https://nejm.ai/3SDd5Rd    Visit http://ai.nejm.org to read all the latest articles on AI and machine learning in clinical medicine. 

#ArtificialIntelligence (AI) holds significant promise for revolutionizing health care by enhancing diagnosis, treatment, and patients’ safety, write the authors of a new Perspective.     However, the current disparity between the abundance of AI research and the scarcity of evidence on real-world impact underscores the urgent need for comprehensive clinical effectiveness evaluations.     These evaluations must go beyond model validation to explore the real-world effectiveness of AI models in clinical settings, especially because so few have gone on to show any meaningful impact. The importance of local context in AI model validation and impact assessment cannot be overstated.     The authors call for increased recognition of implementation science principles and their adoption through development of a network of health care delivery organizations to focus on the clinical effectiveness of AI models in real-world settings to help achieve the shared goal of safer, more effective, and equitable care for all patients.    Read the Perspective “A Call for Artificial Intelligence Implementation Science Centers to Evaluate Clinical Effectiveness” by Christopher Longhurst, MD, MS, Karandeep Singh, MD, MMSc, Aneesh Chopra, MPP, Ashish Atreja, MD, MPH, and John Brownstein, PhD: https://nejm.ai/4f03LjU    #AIinMedicine   

I am thrilled to announce our latest publication in NEJM AI: "FHIR-GPT Enhances Health Interoperability with Large Language Models" led by talented PhD student Yikuan Li. We tackled the challenge of transforming electronic health record data into FHIR resources, a crucial step in advancing health data interoperability. This transformation is often hindered by the heterogeneous structures and formats of health data, especially when critical information is embedded in unstructured data. FHIR-GPT, our innovative solution that leverages the power of large language models to convert clinical narratives into FHIR resources. Our experiments demonstrated that FHIR-GPT achieved an exact match rate of over 90%, surpassing state-of-the-art systems from both academia (e.g., NLP2FHIR) and industry (e.g., Google HNL API). FHIR-GPT also reduced development costs and need for extensive training data and simplified integration of various NLP tools. These findings confirm the incredible potential of LLMs in enhancing health data interoperability, benefiting research, clinical trial support, and public health surveillance. We are proud to contribute to this significant advancement and look forward to seeing the positive impact on the healthcare industry. Let's continue pushing the boundaries of what's possible in healthcare! Full article: https://lnkd.in/ghsYxpm6 Our team included an all-star set of scientists. Full author list: Yikuan Li, Hanyin Wang, Halid Ziya Yerebakan, Yoshihisa Shinagawa & Yuan Luo #HealthData #Interoperability #FHIR #LLM #GPT #Innovation #Healthcare #Research #ClinicalTrials #PublicHealth

A new Perspective summarizes the episode of NEJM AI Grand Rounds where Drs. Andrew Beam and Arjun Manrai hosted Dr. Isaac Kohane.     Dr. Kohane shares his career journey and reflects on the progress of AI in medicine over the past several decades.     The authors also discuss the importance of multidisciplinary training in AI and medicine and the motivation behind the founding of NEJM AI and the Department of Biomedical Informatics at Harvard Medical School.    Read “Why Medicine Must Become a Knowledge-Processing Discipline,” the first Perspective in a series of summaries of NEJM AI Grand Rounds podcast episodes, by Isaac Kohane, MD, PhD, Andrew Beam, PhD, and Arjun Manrai, PhD: https://nejm.ai/4cDiNKm    Learn more about the series: https://nejm.ai/45HhFmG    Listen to the current and past episodes of the podcast: https://lnkd.in/e_PYP-TA    #ArtificialIntelligence #AIinMedicine 

In this episode of the AI Grand Rounds podcast, hosts Arjun Manrai, PhD, and Andrew Beam, PhD, interview Drs. Rohaid Ali and Fatima N. Mirza, a married couple and chief residents at Brown University.     The conversation explores their innovative work applying #ArtificialIntelligence to health care, focusing on two major projects:     📋 Using ChatGPT to simplify surgical consent forms, making them more accessible to patients. This initiative led to widespread adoption within their healthcare system and inspired similar changes in other medical documentation.     🧠 Collaborating with OpenAI's Voice Engine to help a young patient who lost her voice due to a brain tumor by creating a custom AI-generated voice based on a short audio sample.     Drs. Ali and Mirza discuss the challenges and opportunities of integrating AI into medical practice, emphasizing responsible deployment and human oversight. They share insights on balancing personal and professional collaboration as a married couple working on research together.     The episode features a lighthearted “newlywed game” segment, testing how well the couple knows each other’s perspectives.     It concludes with Drs. Ali and Mirza offering advice to early-career doctors interested in AI and sharing their vision for AI’s future in medicine, highlighting the importance of ensuring equitable access to these technologies and the need for thoughtful implementation by medical professionals.    Listen to the full episode hosted by NEJM AI Deputy Editors Arjun Manrai, PhD, and Andrew Beam, PhD: https://nejm.ai/ep20    #AIinMedicine 

The ARISE trial randomly assigned patients to either an AI-powered electrocardiogram (AI-ECG) system or standard care and tested the effect of AI-ECG interpretation and automated text-based short message service notification on treatment delays and diagnostic accuracy for patients with ST-segment elevation myocardial infarction (STEMI).     The AI-ECG system cut the median door-to-balloon time by 14 minutes for patients presenting to the emergency department.     The AI-ECG system also decreased the ECG-to-balloon time by 5.6 minutes for hospitalized patients.     The AI-ECG system also had a high positive and negative predictive value, and the AI-ECG group had fewer STEMI activations in patients not requiring emergent angiography.     However, the single-center design, short follow-up period, and lack of evaluation of care appropriateness and clinical safety end points limit the study’s generalizability.     Although the findings suggest that AI-ECG can expedite STEMI diagnosis and treatment, further research is needed to confirm these results in diverse settings and assess the impact on long-term patient outcomes.    Read the full editorial by Robert Avram, MD, MSc, and William F. Fearon, MD: https://nejm.ai/4bob3dW    And read the full ARISE trial results by Chin Lin, PhD, et al.: https://nejm.ai/4eG7ZwC    #ArtificialIntelligence #AIinMedicine