Three Real Applications of Large Language Models in Nursing

These studies show REAL use cases of LLMs in nursing care and hospitals

Health care leaders are voraciously discussing a significant transformation of care delivery through artificial intelligence, particularly with the emergence of large language models (LLMs) like ChatGPT. These AI systems, trained on vast amounts of text data, can understand and generate human-like responses to complex queries. Think of them as sophisticated pattern recognition systems that have analyzed billions of text examples to learn how to engage in meaningful dialogue and problem-solving. It would be hard to find someone who has not at least heard of Chat GPT these days!

As with any new technology, we must identify the problems that we are seeking to solve before applying it. Good technologies in health services add new capabilities or add value to existing ones thus solving key issues or optimizing operational and clinical performance. But, LLMs are a bit of a capability that has been built without a specific problem to solve. It is like a hammer without a nail, but it is at the same time clear that there are many nails it may be able to hit.

In nursing there are several problems innovators, at present, are looking to solve. First, is workload and burnout issues in the workforce. Nurses are overburdened, understaffed, and quitting their critical patient care roles. In the world of the internet, remote work, and TikTok; nurses can easily find work outside of their traditional roles at the hospital bedside. How can we reimaging the workplace for nurses such that it is competitive and worthwhile?

Second, there is always a desire to improve clinical performance in a cost effective manner. If the application of LLMs can improve clinical quality, reduce adverse events, and lead to better patient outcomes; then they will find a home in health services and nursing care.

Third, organizationally, the hospitals and care delivery organizations that employ nurses are always looking for ways to reduce cost. Safe and effective automation of manual tasks, of which there are many in health care services, may be a useful application of LLM technologies.

Fourth, nursing education is always at a crossroads, it seems, with an undersupply of educators, nursing school slots, and clinical practicum sites. Can LLMs help increase the quality and capacity of nursing education?

Three recent studies highlight promising applications of LLMs in nursing, demonstrating their potential to address critical challenges in education, emergency care, and patient communication. The next five years will yield even more research and use-cases, so stay tuned to Health Tech Happy Hour as we explore these!

Clinical Simulation Training: Building Better Learning Experiences

A study published in Clinical Simulation in Nursing explored using ChatGPT to create realistic training scenarios for nursing students. Traditionally, developing these educational simulations is labor-intensive, requiring 40-80 hours per scenario, according to the authors. The researchers tested ChatGPT's ability to generate five different patient scenarios, including situations like new-onset gestational diabetes and anaphylactic emergencies.

The results were encouraging, but revealed important nuances. While ChatGPT could quickly generate basic scenario frameworks, saving considerable time for educators, human expertise remained crucial. Expert reviewers found the AI-generated scenarios maintained good accuracy, but sometimes missed important clinical details. This suggests a hybrid approach might be optimal – using AI to create initial drafts that experienced educators can then refine. The incorporation of LLMs into clinical software platforms may also be useful to allow for scenario dialog and decision making with real-time feedback for nursing students.

Nursing schools face limitations on graduate-trained nurse educators, and at the same time must increase capacity to meet a growing demand for trained nurses. In recent years, simulation labs have been growing in popularity, but these are expensive and have a finite capacity.

The use of LLM-based scenarios can help supplement the cognitive components of nursing education and training in an engaging and real-world information and decision-making scenario. While there is certainly no replacement of real-world, in hospital experience, these tools can likely help prepare the next generation of nurses.

Emergency Department Triage: Comparing AI Systems

In a more direct clinical application, researchers at a large urban hospital compared how well different LLMs (ChatGPT Plus and Microsoft's Copilot Pro) could perform emergency department triage compared to experienced nurses. This study is particularly relevant given the increasing pressure on emergency departments worldwide, with many facilities struggling to manage growing patient volumes.

The findings were revealing: both AI systems showed comparable overall accuracy to human nurses (around 65%) and actually outperformed humans in identifying high-acuity patients. However, the study highlighted that while the AI systems could accurately assess medical urgency, they could not account for real-world constraints like available beds or staffing levels, which human nurses factor into their triage decisions. This is where further integration of predictive AI models and data from hospital capacity management systems can truly help automate processes and prioritize patient flow.

Automating Patient Communication Through Electronic Health Records

The third study examined using LLMs to draft responses to patient messages in electronic health records (EHR) systems. This addresses a growing challenge in health care – the increasing volume of patient portal messages that can overwhelm clinical staff. According to recent data, about 60% of patients now access their online medical records, up from 25% in 2014¹. This increase in usage creates an overwhelming increase in the number of messages from patients to care teams and has even prompted some health systems to charge for responses.

In this study, LLMs were used to recommend responses to patients and then the clinicians participating in the study decided whether or not to use them. The researchers found that while only 12% of AI-generated draft responses were used without modification, the technology showed promise in specific contexts. Nurses, in particular, found the AI drafts helpful for routine communications, noting that they were appropriately empathetic and helped maintain consistent response quality. Nurses were also more likely to recommend this approach to others with a Net Promoter Score of 58. However, physicians were more hesitant to use the system for complex medical queries requiring nuanced expertise.

Looking Ahead: Promise and Limitations

These studies collectively suggest that LLMs have significant potential in nursing applications, but require careful implementation and robust safety monitoring. Large scale real world evidence will be critical to ensure lower-risk clinical applications with humans in the loop do not produce unintended consequences and actually improve workflows. The technology appears most effective when used as a supportive tool rather than a replacement for human judgment. Key advantages include:

• Reducing time spent on routine tasks

• Maintaining consistency in communications and training materials

• Supporting decision-making in high-pressure situations

However, important limitations remain, including:

• The need for human oversight and expertise (this is often called Human-in-the-Loop AI)

• Challenges in handling complex, nuanced situations (computers are rigid thinkers and while LLMs are lightyears ahead of the computers of yesteryear humans are needed for judgement calls and interpreting issues in the gray areas of medicine)

• The importance of considering real-world operational constraints (electronic medical records have innovation in hospitals locked down and it is costly and challenging to innovate in those scenarios)

As health care continues to face growing demands with limited resources, these AI applications offer promising ways to support nursing professionals while maintaining high standards of care—if real world studies continue to demonstrate their value and benefits over their costs and risks. The key lies in understanding where these tools can best augment human capabilities rather than replace them entirely.

The findings from these studies suggest that the future of nursing will likely involve an increasing partnership between human expertise and AI support tools, with each playing to their respective strengths to improve patient care and outcomes.

1. This is where I add that Judy Faulkner of Epic is dead wrong in her statements about how patient’s do not want access to their own records in an attempt to continue Epic’s monopoly on American’s health information.