Results of cross-sectional study of 50 discharge summaries suggest a large language model can be used to increase their understandability.
A study recently published in JAMA Network Open suggested a large language model (LLM) can be used to make impatient discharge summaries easier to understand for patients.1 Based on understandability scores from 50 discharge summaries and the use of a generative artificial intelligence (AI) platform, the study authors concluded that generative AI could be used to effectively transform these summaries and make them easier to read.
“By law, patients have immediate access to discharge notes in their medical records. Technical language and abbreviations make notes difficult to read and understand for a typical patient. Large language models (LLMs [eg, GPT-4]) have the potential to transform these notes into patient-friendly language and format,” the authors wrote.
The study investigators evaluated discharge summaries for 50 patients from the General Internal Medicine service at New York University (NYU) Langone Health from June 1-30, 2023, for readability and understandability. Readability was measured using the Flesch-Kincaid Grade Level and understandability using Patient Education Materials Assessment Tool (PEMAT) scores. The original summaries were then compared to the new summaries generated by the LLM, Microsoft Azure OpenAI. The new summaries were processed into a patient-friendly format between July 26 and August 5, 2023.
The results of the study greatly favored the summaries generated by AI, according to the investigators. Mean Flesch-Kincaid Grade Level was significantly lower in the patient-friendly discharge summaries. Additionally, PEMAT understandability scores were significantly higher for patient-friendly discharge summaries.
“We found the patient-friendly discharge summaries had fewer words than the original discharge summary, a difference that was statistically significant. Readability as measured by the Flesch-Kincaid Reading Ease score was significantly higher in the patient-friendly discharge summaries compared with the original discharge summaries,” the authors wrote. “Conversely, Flesch-Kincaid Grade Level was significantly improved (ie, was lower) for patient-friendly discharge summaries.”
The authors also found a large difference in understandability scores between the original summaries and the patient-friendly discharge summaries based on PEMAT. Scores marked the patient-friendly summaries at 81% compared to just 13% for the originals.
“We were able to show that our patient-friendly discharge summaries were consistently at a sixth or seventh grade reading level. This is markedly different from original discharge summaries, in which reading ease varies widely but is typically at the 11th grade reading level,” the authors wrote. “We measured understandability using the PEMAT instrument and found patient-friendly discharge summaries to have high levels of understandability. This is also a marked difference from original discharge summaries.”
The authors identified a few limitations to this study, but said they can be used as ways to improve future research. The patient-friendly discharge summaries created by the LLM were limited to the English language, which is a barrier to transparent, equitable care of non-English-speaking patients and caregivers. Another limitation is the lack of validated generative AI instruments in healthcare. The authors developed an original instrument for this study, but look forward to including human-generated patient-friendly discharge summaries to further validate the AI.
“In this cross-sectional study, the strong performance of our generative AI prompt in creating highly readable and understandable discharge summaries shows the promise of using generative AI to make patient information more accessible to patients themselves,” the authors concluded.
1. Zaretsky J, Kim JM, Baskharoun S, et al. Generative Artificial Intelligence to Transform Inpatient Discharge Summaries to Patient-Friendly Language and Format. JAMA Netw Open. 2024;7(3):e240357. doi:10.1001/jamanetworkopen.2024.0357
