Fully autonomous primary care's ready to be launched. Apparently.

As evidence based as it sounds.

It’s funny what turns up in the media and what doesn’t.

Healthtech really isn’t great for this. You’d be forgiven for thinking that the only companies that existed were making consumer wearables. One of the best things about starting Healthtech Pigeon has been surfacing lesser known, lesser (let’s just say) ‘appreciated’ news. Here’s a corker…

In February, Mount Sinai ran the first independent safety test of ChatGPT Health. Among genuine emergencies (and remember, OpenAI fields questions from about 40 million people a day), it under-triaged 52% of cases. It looked at diabetic ketoacidosis (DKA) and impending respiratory failure and suggested people wait a day or two before seeing anyone. Cool…

Then a month later, Curai Health (US-based virtual primary and urgent care company) published a serious, real-world study of its own system across 2,379 patient ‘encounters,’ and used it to argue that AI is now ready to run defined parts of a primary care visit without a doctor in the loop. Really?

If you want the horses mouth (abstract)…

“Here, we report, to our knowledge, the first large-scale, clinician-blinded, real-world evaluation of a multi-agent LLM-based system deployed within a nationwide U.S. primary care telemedicine platform, assessing readiness for task-specific autonomous deployment… Our findings provide the first real-world evidence of readiness for safe autonomous clinical AI…”

_{Saenz, Agustina & Schumacher, Elliot & Naik, Dhruv & Khosla, Neal & Kannan, Anitha. (2026). From Concept to Clinic: Real World Evidence for Autonomous AI Deployment in Primary Care Telemedicine. 10.64898/2026.03.18.26348749.}

Their case is that an engineered stack with hard-coded safety checks around the language model is now safe enough to go at a set of common physical conditions, and the FDA’s invited to rewrite its rules to allow it… Interesting.

Now, if you’ve read Pigeon for a while, I can hear you typing “hang on, you were perfectly happy with the likes of Limbic that have a similar stack for patient-facing mental health triage. Why bring this up?” Good question. Yes, I was, I still am. When it comes to patient-facing AI, I’ve learned that detail and nuance is absolutely everything, so allow me to explain…

When AI does one bounded job, i.e. sorting people into the conditions, and it’s built on structured instruments like the PHQ-9 and GAD-7, it augments a clinician’s assessment, and you earn Class IIa medical device status through a regulatory route that already exists. However, when you make a diagnostic call across open-ended primary care, arguing the doctor can step out of the loop… and ask the FDA to build it a new pathway and fresh liability law for when there’s no clinician left to hold responsible, it’s… well… kinda different.

In mental health triage there's usually a clinician downstream and time to correct a miss, and the AI runs inside a supervised pathway, not instead of one.

In physical medicine in primary and urgent care, you might not get a clinician downstream or the gift of time before a patient deteriorates. DKA might present initially as a stomach bug, an ectopic might look like a UTI for a few hours, and the window to catch these before a rapid decline can be very short.

I’m not actually arguing that humans are better than AI at catching that kind of thing or vice versa (although, I’m pretty sure if we were choosing a reality it would be AI+humans rather than either, alone), I’m saying that when a doctor gets it wrong, there's a history you can reconstruct, a duty of care, a regulator, and a coroner who can ask why those questions and not others. Take the clinician out, as is being suggested here, and, ok the error rate might not change, but everything built to account for the error’s gone. What you’re left with is very weird at best, and I’ll let you add a word to describe the situation that, god forbid, you’re bereaved.

So here we are. Tens of millions of people using LLMs to decide whether their symptoms are an emergency. Companies publishing papers arguing they’re ready to take the clinician out of the loop for defined categories of care, asking regulators to ‘please sort.’ And, I’m just wondering… who actually decided that the autonomous diagnosis road was the one anyone wanted us to travel down?

Well… I need to show you something…