A Hartford Healthcare building in downtown Hartford, Connecticut, on November 8, 2023. Getty Images/JHVE Photo

...

Alexis Kayser

By Alexis Kayser

Healthcare Editor

0

Share

Newsweek is a Trust Project member

See more of our trusted coverage when you search.Prefer Newsweek on Googleto see more of our trusted coverage when you search.


This is a preview of the March 26 edition of Access Health—Tap here to get this newsletter delivered straight to your inbox on Thursday mornings.


Good morning. We finally have data on how patients are using consumer AI chatbots—and it didn’t come from the LLMs’ parent companies!

Read More on Health Care

Here are a few key data points from KFF’s latest Tracking Poll on Health Information and Trust, which came out yesterday:

And—the kicker—although 77 percent of users expressed privacy concerns, 41 percent said they’ve uploaded personal medical information into an AI chatbot.

When I saw those numbers juxtaposed, I was reminded of a Newsweek cover story I wrote back in 2024, examining the extent to which AI could relieve physician burnout. Many of the doctors I spoke with said they disliked the term “burnout,” which feels, to them, like a simplification. “Burnout” implies that they struggled to manage the workload. “Moral injury,” however, accounts for the reason why: a mismatch between the level of care clinicians want to provide and the level of care that they’re capable of providing in a system that is out of their control. The latter term boomerangs responsibility to the institutions of health care rather than the individual providers who compose them.

When people work hard and play by the rules, but still don’t feel capable of making a real impact, they’re much more likely to experience the symptoms associated with burnout: exhaustion, cynicism, detachment, a sense of ineffectiveness.

It may be tempting to attribute the stats above to some individual lapse of judgement, overreliance on technology or addiction to the convenience economy.

But I thought that Dr. Atiq Bhatti—the faculty administrator and clinical research operations lead in the department of medicine at Eastern Virginia Medical School and Virginia Health Sciences—had a more nuanced take during our March 5 webinar on answering patients’ questions before the internet does.

The public desire to ask AI for health care recommendations points to “a design failure in our health care system,” Bhatti said. “It’s not a patient behavior problem…they have an absolute right to look for that information.”

“We have always talked about ambient tools for burnout of the physicians,” he said. “Let's talk about the burnout of a patient.”

Health care leaders from across the industry concur that the system is imperfect (I hear the term “broken” more frequently). And—as KFF’s data points on affordability, comfort level and response time illustrate—patients are seeking something from consumer AI tools that they are not getting from their providers.

The fact that folks are willing to upload medical records to LLMs despite their privacy concerns is troubling. It doesn’t compare to the moral injury that veterans and physicians experience, but there’s an undeniable echo of that value misalignment. Patients might not fully trust ChatGPT, but they’ve given a myriad of reasons why they trust it more than their health system.

It’s exciting to see health systems recognize this and work to provide alternatives. Just this morning, Hartford HealthCare and KHealth unveiled their “PatientGPT” tool, a generative AI chatbot that integrates with Epic and can answer patients’ inquiries in MyChart. Unlike many of the consumer-grade tools, PatientGPT can escalate to a provider 24/7. (Read on to the Pulse Check section for my interview with execs from both organizations).

Still, I wonder about the 65 percent of patients who wanted a “quick” answer. Will they run into a wall when their health system’s LLM cuts off the conversation, to be continued at a specialist appointment that is months away? Or the 36 percent of patients who felt more comfortable talking to AI than a human doctor—will they give up that immediate sense of privacy for the promise of HIPAA-level security?

And what about the 20 percent who use AI because they can’t afford a doctor’s visit? How many of them have a MyChart account? How many will fall off when the answer comes at a cost?

I’m sure we’ll see that dataset someday, but in the meantime, I want to know what you think. You can reach me at a.kayser@newsweek.com.


In Other News

Major health care headlines from the week


Pulse Check

Executive perspectives on key industry issues

Dr. Padmanabhan Premkumar, MD, VP

...

b344e1c5-aef2-4d9e-b4a1-ffcfd12d4349

...

1 of 2

Yesterday, I sat down with Dr. Padmanabhan Premkumar, president of Hartford HealthCare Medical Group and Ran Shaul, co-founder and chief product officer of K Health ahead of today’s PatientGPT launch at the Connecticut health system.

PatientGPT is a LLM developed by K Health that is embedded directly within the health care network and answers patients’ medical inquiries. Clinical teams have full visibility over the chat logs, and the AI tool can escalate to live providers 24/7, according to the duo.

Read on for a portion of my conversation with Premkumar and Shaul, covering how the tool works, what early studies have shown and how the two companies expect it to benefit both patients and providers.

Editor’s Note: Responses have been edited for length and clarity.

To start, tell me about the idea behind PatientGPT. Where did it originate, and how long have you been working on this product?

Shaul: We’ve been building clinical AI at K Health for about nine years. It’s a language model, but fundamentally different from general-purpose models like Gemini—it’s trained on anonymized EMR data.

About six months ago, Prem and I sat down with leadership at Hartford HealthCare and realized something important: patients are increasingly seeking advice from general-purpose LLMs. That’s not new—remember “Dr. Google?”—but the depth has changed. Patients are now uploading medical records and lab results into tools that aren’t secure or connected to their care.

We saw that patients actually want this kind of interaction—but the experience today is fragmented. You ask a question, get a long answer, but it’s not connected to your care team. Then you copy and paste it into a message to your doctor. It’s friction.

So we said: let’s build this inside the health system. A clinically trained AI, connected to your medical record, embedded in your care team, and available 24/7. If you’re unsure about something, you can immediately escalate to a provider. That’s the core idea—we’re responding to patient demand for accessible, trusted care.

How did you determine that patients were using general LLMs more frequently?

Shaul: We see thousands of patients daily on our platform. Many journeys start with questions that clearly originated from general LLMs. We’ve also seen increased in-basket traffic, patients copying and pasting responses into EHR messaging systems.

And external data supports this.

Premkumar: Over 230 million people are using ChatGPT for health advice weekly.

How are physicians affected by this increase in LLM-generated inquiries?

Premkumar: The health care journey is fragmented. Patients seek information from multiple sources, but ultimately trust their physicians. Today, they send messages, upload information and ask questions, but it’s difficult for providers to interpret intent.

PatientGPT helps us understand what patients are actually asking. It summarizes, categorizes and routes questions appropriately. That makes providers’ jobs easier and improves the patient experience.

Health care should feel more like a consumer experience: simple, accessible, responsive. That’s what we’re trying to build.

How would you explain the difference between PatientGPT and this new class of health-focused consumer AI tools, like Copilot Health?

Premkumar: The key difference is where the product lives. PatientGPT is embedded directly in the patient portal. Patients use their existing MyChart login. They’re not uploading sensitive information to an external platform.

Second, it’s connected to the care team—primary care, specialists, everyone. External tools don’t have that context.

Third, HIPAA and data protection are foundational. That’s why it stays within the health system environment.

Shaul: General-purpose LLMs are disconnected from care teams. They don’t know your history, your appointments, or your providers.

We designed PatientGPT with a “human-in-the-loop” approach. It understands your context within the health system and connects directly to your care team. That’s a fundamental difference.

Can care teams access patients’ chat histories at any time, or do patients have to give explicit permission for their providers to see their inquiries?

Premkumar: Yes, we transmit the conversation into the EMR [Epic]. We can escalate the conversation to their primary care or specialty care physician. We can transmit messages directly from the conversation in the patient portal into providers’ in-baskets.

We don't see this as living outside of the care team. We see this as a connected experience for our patients.

A Hartford Healthcare building in downtown Hartford, Connecticut, on November 8, 2023. (Getty Images/JHVE Photo)

...

Hartford shared an internal, IRB-reviewed study of PatientGPT that found it to make nearly 90 percent fewer high-risk clinical errors than ChatGPT. Can you explain what “high-risk failure rate” means in this instance?

Shaul: A high-risk failure occurs when the AI misses a critical situation. For example, if someone reports chest pain, the correct response is immediate escalation to the ER.

Studies have shown general models fail to recognize these situations about 52 percent of the time. In our testing, we reduced that to 5.9 percent.

The key difference is design. Instead of always giving answers, PatientGPT asks follow-up questions and escalates to a human when needed. In ambiguous cases, we don’t guess. We involve a provider.

How are escalations handled?

Premkumar: We have multiple pathways. Emergency cases go through standard emergency protocols.

For other needs, we can connect patients to virtual care 24/7. We’ve already treated nearly 50,000 patients this way through HHC 24/7, which we created with K Health last year.

We can also route to primary or specialty care. The goal is to connect patients to the right resource at the right time.

How was the model trained?

Shaul: It wasn’t trained specifically on Hartford data. We created an algorithm that has a proprietary language model that was trained on data that K Health has access to. It’s our own patient information. We have done 4 million medical consultations over the years.

We also contract with the Mayo Clinic, and we are part of the Mayo Clinic vendor network that can access their de-identified information.

Can PatientGPT be replicated at other health systems?

Shaul: Yes. Health systems are recognizing that patients expect real-time, trusted information. Leaders across the country are asking the same questions: how do we stay part of the patient journey outside the clinic?

We believe this model can scale.

What metrics will define success in the first year?

Premkumar: First, consumer experience. Do patients trust and use it?

Second, quality and safety.

Third, provider experience. Does this make their workflows better?

Fourth, insights. What are patients asking for, and where are we falling short as a system?

Shaul: We’re also looking at broader impact: better understanding of care plans, improved adherence, stronger preventive care and better chronic disease management. Much of health care happens between visits, and that’s where this can have real impact.

Are you concerned that existing bottlenecks within the health system could limit patient adoption of and/or satisfaction with PatientGPT?

For example, say the model recognizes that the patient needs to speak with a professional and ends the conversation, escalating the issue to a specialty like neurology. It takes more than a month, on average, to get a neurology appointment.

What's stopping a patient from migrating to a general purpose LLM and trying to get more immediate answers to their questions, if that’s what they’re after?

Premkumar: I think that’s the beauty of what we’re trying to create. As the person who leads the medical group here, I want to know where my choke points are. If I need to have more neurology, cardiology or urology access, that’s a problem that I want to be able to solve.

If you go to ChatGPT, you may get an answer to your question, but you have no idea whether that’s the right question. Ultimately, you still have to see the neurologist—and I still have to create access to that neurologist.

I think that things like PatientGPT will force health care systems to be better.


C-Suite Shuffles

Where health care leaders are coming and going


This is a preview of the March 26 edition of Access Health—Tap here to get this newsletter delivered straight to your inbox on Thursday mornings.

View 0 Comments

Request Reprint & Licensing

Submit Correction

View Editorial & AI Guidelines