Using LLMs and the Clinical Care Classification System Interview for aligning EHR flowsheets: An interview with Hao Fan from the Washington University School of Medicine.

We would like to thank Hao Fan, a PhD candidate in the Biomedical Informatics and Data Science Program at Washington University School of Medicine, St. Louis, MO for answering some of our questions about their research which involves using the Clinical Care Classification System:

What problem were you trying to tackle with this research?

We developed a semi-automatic pipeline for aligning electronic health record (EHR) flowsheets across multiple healthcare organizations. EHR flowsheet alignment is a challenge in data harmonization due to the redundancy in flowsheet structures and the variations in clinical contexts and nursing practices among institutions.

What was your approach?

The semi-automatic pipeline we developed (see figure below) aligns flowsheet template-measure (T-M) pairs using exact, lexical, and semantic matching techniques. We leveraged large language models (LLMs) to facilitate semantic alignment by mapping T-M pairs to the Clinical Care Classifications (CCC) terminology. To improve concept ranking and ensure reliability, we assessed the consistency of LLM-recommended concepts within the CCC hierarchy.

Why did you choose to use the CCC?

We selected the CCC terminology because it provides a single unified framework that directly links nursing diagnoses, interventions, and outcomes. Additionally, CCC has been mapped to SNOMED-CT, making it a valuable bridge for enhancing interoperability across different healthcare systems.

What were your findings?

Our results showed that the semi-automatic pipeline achieved a 63% flowsheet alignment rate with a 53% concept mapping rate when restricted to the top-ranked CCC concept. When expanding the mapping to the top three CCC concepts, the pipeline improved alignment to 96.5% and concept mapping to 96%. These findings demonstrate the potential of leveraging artificial intelligence (AI)-driven approaches to enhance data harmonization across healthcare organizations.

What are your next steps?

Our next step is to develop a web-based interface that will allow domain experts to manually review and validate the mapping results. This platform will facilitate expert verification, ensuring greater accuracy and reliability in flowsheet alignment. By incorporating human expertise into the process, we aim to refine our methodology and enhance its practical utility for real-world applications.

Any advice to other people doing research in this area?

Researchers working on concept mapping and data harmonization can benefit from integrating AI techniques into their workflows. LLM-based methods can significantly improve the effectiveness and efficiency of traditional mapping approaches. Moreover, human validation remains essential to ensure clinical accuracy in healthcare data harmonization.

Who were the members of your research team and funding source?

I am working with Dr. Po-Yin Yen, Associate Professor of Medicine at Washington University School of Medicine, St. Louis, MO and Dr. Sarah C. Rossetti, Associate Professor of Biomedical Informatics and Nursing at Columbia University.  Our funding source is Agency for Healthcare Research and Quality (AHRQ): R01HS028454

Where can people learn more about this work?

Hao Fan, Sarah C Rossetti, Jennifer Thate, Rosemary Mugoya, Albert M Lai, Po-Yin Yen, Semi-automated pipeline to accelerate multi-site flowsheet alignment and concept mapping in electronic health records, Journal of the American Medical Informatics Association, 2025; ocaf076, https://doi.org/10.1093/jamia/ocaf076

Thanks so much for your time. Where can readers find you if they have further questions?

Thanks for having me! People can contact me using my academic email: fanh [at] wustl.edu