In the latest episode of the NEJM AI Grand Rounds podcast, hosts Arjun Manrai, PhD, and Andrew Beam, PhD, interview Vijay Pande, PhD, a general partner at Andreessen Horowitz (A16Z) where he leads investments in health care and life sciences. The conversation explores Dr. Pande’s journey from academia to venture capital, his views on the future of #ArtificialIntelligence in health care and biomedicine, and insights into the investment landscape for biotech and health tech companies. Dr. Pande discusses the challenges and opportunities in integrating AI into medical practice, the potential for AI to democratize health care access, and his thoughts on the development of artificial general intelligence. Listen to the full episode hosted by NEJM AI Deputy Editors Arjun Manrai, PhD, and Andrew Beam, PhD: https://nejm.ai/ep21 #AIinMedicine
NEJM AI
Book and Periodical Publishing
Waltham, Massachusetts 11,044 followers
AI is transforming clinical practice. Are you ready?
About us
NEJM AI, a new monthly journal from NEJM Group, is the first publication to engage both clinical and technology innovators in applying the rigorous research and publishing standards of the New England Journal of Medicine to evaluate the promises and pitfalls of clinical applications of AI. NEJM AI is leading the way in establishing a stronger evidence base for clinical AI while facilitating dialogue among all parties with a stake in these emerging technologies. We invite you to join your peers on this journey.
- Website
-
https://ai.nejm.org/
External link for NEJM AI
- Industry
- Book and Periodical Publishing
- Company size
- 201-500 employees
- Headquarters
- Waltham, Massachusetts
- Founded
- 2023
- Specialties
- medical education and public health
Updates
-
An article by J. Fajtl et al. addresses the challenge of implementing algorithms in health care screening by proposing a methodology for independent evaluation, demonstrated through a large-scale diabetic eye screening program. Automated retinal image analysis systems (ARIAS), including #ArtificialIntelligence for detecting diabetic retinopathy (DR), were evaluated for their technical and operational viability. Twenty-five vendors were invited, and eight participated, processing 1.2 million images from 202,886 encounters, representing diverse ethnic groups. The evaluation focused on detecting severe DR across ethnic subgroups without access to human grading data. The findings highlight the importance of transparent, equitable, and robust evaluation of clinical AI in screening programs to establish reliable health care standards before deployment. Learn more in the Original Article “Trustworthy Evaluation of Clinical AI for Analysis of Medical Images in Diverse Populations” by J. Fajtl et al.: https://nejm.ai/3Arq3es #AIinMedicine
-
-
Traditional scientific methods may not be sufficient to evaluate generative #ArtificialIntelligence (GenAI) due to its nondeterministic performance and lack of structural transparency. Viewing GenAI as an ecosystem may facilitate global performance analysis, such as estimating resilience and sustainability as conditions or tasks change. Adopting GenAI necessitates a shift in evaluation focus from reliance on premarket assessment to real-world postmarket surveillance. Read the Perspective “AI as an Ecosystem — Ensuring Generative AI Is Safe and Effective” by Enrico Coiera, MBBS, PhD, and David Fraile Navarro, MBBS, PhD: https://nejm.ai/46GCkaI #AIinMedicine
-
-
A new Perspective summarizes the episode of NEJM AI Grand Rounds in which Dr. Adam Rodman joined cohosts Drs. Arjun Manrai and Andrew Beam for a wide-ranging conversation about the history and future of medical diagnosis. Drawing on his experience as a historian of medical epistemology and clinical reasoning — that is, how doctors “know” things about diseases and their patients — Dr. Rodman places our current discussion about the diagnostic abilities of large language models (LLMs) into a century-long context of attempts to build diagnostic #ArtificialIntelligence. Dr. Rodman shares his surprise at the diagnostic reasoning abilities of LLMs and his own research on clinical evaluations of the GPT-4 model and its ability to change human reasoning. The authors discuss the implications of these technologies for future medical practice and medical education, with optimism that these technologies might be able to preserve — or really rediscover — the most human elements of medicine. Read the Perspective “Exploring the Past — and Future — of Medical Diagnosis and Artificial Intelligence” by Adam Rodman, MD, Andrew Beam, PhD, and Arjun Manrai, PhD: https://nejm.ai/4d6nE7n Listen to the full episode of the podcast: https://nejm.ai/ep19 #MedicalAI
-
-
Volume 1, No. 9 is now available! Here are the latest articles available in the September issue of NEJM AI: Save this post to revisit later (click the 💬 button at top right of post). 🤖 Editorial: AI-Based Diabetic Macular Edema Screening for Improved Care in Low-Resource Settings https://nejm.ai/3YRFtCL 🛡️ Perspective: AI as an Ecosystem — Ensuring Generative AI Is Safe and Effective https://nejm.ai/46GCkaI 🔍 Perspective: Exploring the Past — and Future — of Medical Diagnosis and Artificial Intelligence https://nejm.ai/4d6nE7n 📈 Original Article: Advancing Diabetic Macular Edema Detection from 3D Optical Coherence Tomography Scans: Integrating Privacy-Preserving AI and Generalizability Techniques — A Prospective Validation in Vietnam https://nejm.ai/3AKqBvV 🌍 Original Article: Trustworthy Evaluation of Clinical AI for Analysis of Medical Images in Diverse Populations https://nejm.ai/3Arq3es 🛠️ Case Study: Evaluation of AI Solutions in Health Care Organizations — The OPTICA Tool https://nejm.ai/4dnu3LH 💻 Case Study: GPT-4 Performance, Nondeterminism, and Drift in Genetic Literature Review https://nejm.ai/4ckgafG Visit http://ai.nejm.org to read all the latest articles on AI and machine learning in clinical medicine. #ArtificialIntelligence #AIinMedicine
-
-
Large language models (LLMs) have potential for enhancing literature reviews in clinical genetic testing. S.J. Aronson et al. evaluated the GPT-4 models for their effectiveness, consistency, and drift over time in classifying functional genetic evidence from literature. Using a two-prompt sequence optimized with 45 article–variant pairs, GPT-4 first identified functional evidence in articles, then classified it as pathogenic, benign, or intermediate/inconclusive. Testing with 72 manually classified pairs from December 2023 to February 2024 revealed significant variability in results due to nondeterminism and drift, which improved after January 18, 2024. The final 20 runs starting January 22, 2024, showed the following performance: initial prompt with 92.2% sensitivity, 95.6% PPV, and 86.3% NPV; pathogenic evidence detection with 90.0% sensitivity, 74.0% PPV, and 95.3% NPV; and benign evidence detection with 88.0% sensitivity, 76.6% PPV, and 96.9% NPV. These findings highlight the importance of monitoring nondeterminism and drift when incorporating LLMs into clinical workflows, as unaddressed variability could affect patient care. While the prompts are useful for prioritizing articles, they are not yet reliable for automated decision-making, indicating the need for further improvements. Learn more in the Case Study “GPT-4 Performance, Nondeterminism, and Drift in Genetic Literature Review” by S.J. Aronson et al.: https://nejm.ai/4ckgafG #ArtificialIntelligence #AIinMedicine
-
-
Drs. Rohaid Ali and Fatima N. Mirza discuss the importance of equitable access to #AI in #healthcare, and the responsibilities of future physicians in implementing and studying these tools to benefit all patients. Listen to the full episode hosted by NEJM AI Deputy Editors Arjun Manrai, PhD, and Andrew Beam, PhD: https://nejm.ai/ep20 #medicine
-
Read "GPT-4 Performance, Nondeterminism, and Drift in Genetic Literature Review," a new Case Study by Sandy Aronson and colleagues: https://nejm.ai/4ckgafG
Generative AI inconsistency can present a risk to clinical processes that should be considered in hazard analyzes. This may be an obvious statement, but it appears to me that the risk of inconsistency increases with task complexity. This figure from our new @MassGeneralBrigham study shows metrics generated through repeated runs of a dataset focused on testing GPT-4 series models’ ability to assess evidence of variant pathogenicity contained in genetic literature. In my view the models perform surprising well, but they are also surprisingly inconsistent. It was great fun to work with Matt Lebo, PhD, FACMG, Kalotina Machini, Jiyeon Shin, Pranav Sriraman, Sean Hamill, Emma H., Charlotte Mailly, Angie N., Sami Amr and Michael Oates on this analysis. Thank you NEJM AI for helping us spread the word on this issue! Note to API users: The risk of inconsistency is present even when steps are taken to reduce inconsistency including setting the temperature parameter to 0, setting seed to a constant value, and monitoring the fingerprint.
-
-
Advancing health data interoperability can significantly benefit research, including phenotyping, clinical trial support, and public health surveillance. Federal agencies such as the Office of the National Coordinator of Health Information Technology, the Centers for Disease Control and Prevention, and the Centers for Medicare & Medicaid Services are collectively promoting interoperability by adopting the Fast Healthcare Interoperability Resources (FHIR) standard. However, the heterogeneous structures and formats of health data present challenges when transforming electronic health record data into FHIR resources. This challenge is exacerbated when critical health information is embedded in unstructured rather than structured data formats. Previous studies relied on separate rule-based or deep learning–based natural language processing (NLP) tools to complete the FHIR transformation, leading to high development costs, the need for extensive training data, and the complex integration of various NLP tools. In this study, the authors assessed the ability of large language models (LLMs) to convert clinical narratives into FHIR resources. The FHIR–generative pretrained transformer (GPT) was developed specifically for the transformation of clinical texts into FHIR medication statements. In experiments involving 3671 snippets of clinical texts, FHIR-GPT achieved an exact match rate of more than 90%, surpassing the performance of existing methods. FHIR-GPT improved the exact match rates of existing NLP pipelines by 3% for routes, 12% for dose quantities, 35% for reasons, 42% for forms, and more than 50% for timing schedules. These findings provide confirmation of the potential for leveraging LLMs to enhance health data interoperability. Read the Case Study “FHIR-GPT Enhances Health Interoperability with Large Language Models” by Yikuan Li, MS, Hanyin Wang, PhD, Halid Ziya Yerebakan, PhD, Yoshihisa Shinagawa, PhD, and Yuan Luo, PhD: https://nejm.ai/3SDd5Rd #ArtificialIntelligence # #HealthIT
-
-
Data standards for the health information ecosystem have played a critical role in enabling software integration across health care enterprises for data sharing, analysis, clinical research, and public health. However, the ability to use large language models (LLMs) to dynamically extract unstructured data into a standardized form for downstream use poses a question about the future of health data. Namely, what role do data standards play in the era of LLMs, and will we need data standards at all? Gabriel Brat, MD, MPH, FACS, Josh Mandel, MD, and Matthew B.A. McDermott, PhD, address this question in a new editorial. Read “Do We Need Data Standards in the Era of Large Language Models?”: https://nejm.ai/4dcpyDb #HealthIT #AIinMedicine
-