Kaiser nurses say AI, surveillance are making their jobs and patient care worse
localnewsmatters.org518 points by gnabgib 13 hours ago
518 points by gnabgib 13 hours ago
I RFTA and the majority of the complaints are about call center metrics and the pressure to ration care. These are real concerns about misuse of metrics, but not AI. The AI empathy thing was a 2024 pilot that was discontinued.
FWIW my wife works for Kaiser and finds a lot of value in the the medical LLM tools available to her. She tells me being able to do live translation, summarize notes, and quickly get comprehensive answers save her time and help her give better care. Her older patients also frequently come in bringing AI-powered alerts from their apple watches that detected cardiac events.
It's annoying that we use broad terms to describe a set of technologies that in some ways can be problematic and in another ways are very beneficial. We gotta evaluate each of these as they come rather than talk about blanket bans.
Being close relative for several med and care workers we have discussed it a lot and consensus is that it really depends. For example relying on LLM summaries sounds great until it doesn't. It doesn't matter whether you misunderstand LLM summary or LLM "misunderstands" you – there are real risks involved, and you wouldn't want them to weigh on your conscience if they were to materialize.
Relying on LLM to summarize things for you has one more issue. To outsiders, this seems like a tedious process, but is actually very important part of the thought process. Wording your thoughts and writing these down helps people to discover new aspects of the problem. It's how people learn.
At the moment consensus is that it must not be banned, but also not mandated in any way - people must take responsibility, and they must be able to decide for themselves where and when the LLM use is justified and where it is not.
I don’t think that it is possible to both allow the use of LLM and not mandate them in modern metric driven work places. Either you ban them or you force people to use them for game theoretic reasons: they are slower than their peers and quality of the work is harder to measure than quantity. All you achieve is shifting the blame to the employees if the LLM messes up. Come to think of it, that probably is a highly desirable outcome for the decision makers, so perhaps that will actually be the policy that becomes universally adopted.
> I don’t think that it is possible to both allow the use of LLM and not mandate them in modern metric driven work places.
What I’d personally be most concerned about would be the risk of bad models instead of SOTA being used which would be especially error and hallucination prone.
If you’re gonna do it, do it right. Otherwise don’t bother at all.
> All you achieve is shifting the blame to the employees if the LLM messes up.
How is this not a good thing for everyone?
> where and when the LLM use is justified and where it is not
while being bombarded with articles like "AI makes things worse", "AI consumes all the water" and the like
Things like this are (sadly) common (and age-old) problems with automation and computerization. (For a vivid account of this phenomenon, check out the novel _Close to the Machine_, by Ellen Ullman.)
As executives and analysts increasingly use the "AI" craze to push automation and computerization (and layoffs) generally, even aside from AI proper, it should not be surprising that the individuals and groups opposing those moves also use the same labels.
The lack of precision in language here sucks. It sucks for the discourse and it also sucks when it comes to focusing anger and productive energy on the core problems (obfuscation of human responsibility, erosion of human agency, declining institutional flexibility, deprofessionalization, etc.). But it doesn't begin with the critics of AI.
>The lack of precision in language here sucks.
It's a feature. Or at least, a perk. If they want to claim this new shiny rock is AI and people buy it, then of course it's in their best interest to keep the black box mysterious. Being subterfuge for muddying the discourse of critique is just a nice side bonus.
But it is only a perk for the scam artists who benefit from that.
Yes, it makes sense that the confusion aligns with their interests, and they are unavoidably a big part of the conversation. But it remains a problem for the non-overlapping group of people who actually value the social contract, and for us finding a solution which helps take one more step to defeat the scammers remains valuable.
Near as I can tell, like crypto, 90% of the ‘discussion’ has been taken over by scam artists, and any folks trying to have a non-scam discussion get yelled at by everyone else.
I am very interested in your reading of Close to the Machine. I read it myself a couple of years ago and found it a wonderful telling of the early days of tech, with overtones of the "technology workplace" that were still very true to this day. I did not pick up on any commentary on automation or computerization, outside of the general critique of bureaucratic systems that alienate you from the outcomes of your labor.
Do you have anything I could read to understand your reading better? I would love to be able to dive back into one of my favorite books with a new lead.
> AI-powered alerts from their apple watches that detected cardiac events
Surely these are “good old-fashioned AI” (statistical learning) and not LLM, though.
I just want to be clear that the “medical LLM” tools are the new ones, and the Apple Watch alerts aren’t.
LLMs are statistical learning. GOFAI is symbolic, rules-based stuff, expert systems and that.
There is still a categorical difference between how they are being used. Specifically analytic vs generative. Generative AI (LLMs and image generators) are the ones people have issues with - pretty much nobody cares about ML processing for analysis.
There’s a bit of a grey area, for example speech recognition. Would you classify that as analytic or generative? Whisper and speech LLMs work pretty well, but can completely make up stuff that wasn’t in the audio at all (see e.g. “thank you for watching” transcribed during silence). Other approaches are closer to the acoustic evidence but may make other mistakes (especially wrongly transcribing long tail, low frequency terms). Pick your poison.
> pretty much nobody cares about ML processing for analysis.
I work in a bank and a can tell you that the customers absolutely hate ML when it rejects their loan application. Over the pond in the US, I have an impression that the fico score is not exactly popular either, but I have no first hand experience.
As long as you can get a reason from the model, it's not that bad.
Black box automatic decision making is much more problematic.
Cardiac events from Apple Watches is not “AI” though
It unequivocally is AI. It's just not LLM-powered.
The rising LLM = AI equivalency is unfortunate.
It's machine learning, which has overlap with AI but is not completely equivalent.
The “overlap” is that all machine learning is AI, but not all AI is machine learning.
> but not all AI is machine learning
I will instead pick at this latter part of your claim. What is an example of something that is AI but that is not ML..?
> The “overlap” is that all machine learning is AI ...
"All machine learning" is not AI, as k-means clustering and linear regression, amongst others, are very much ML without qualifying as AI algorithms.
https://en.wikipedia.org/wiki/Artificial_intelligence
As it is taught literally every single AI/machine learning course on the world, machine learning is very much part of AI completely since inception.
I don’t completely understand why it is this important for you to argue against this completely defined fact.
It is correct to argue about misleading terminology. "AI" contains the word "intelligence", and for instance logistic regression algorithm is not intelligent, while it is clearly ML, since machine learns something. As Machine learning is broader category, it should include Artificial Intelligence, not vice versa.
Also, 'every single course' is perhaps an overstatement - a course that I co-authored tries to get it right from the first principles.
You're just making up your own definitions. Have at it, but as you've been told: this stuff is not new.
Technically linear regression is statistics rather than ML, but I feel like the GNU/Linux people whenever I point that out.
The machine is learning something so that it can produce outputs based on its learned knowledge. At a high level that seems to be very clearly AI. What am I missing here? You’re probably right, I’m asking genuinely.
It's a matter of definitions, but I can at least understand someone wanting to make a distinction between reactive and non-reactive 'AI' (such as data filters).
There's overlap and edge cases, though: Maybe you have a program that summarizes texts. One could argue that's no different from a passive filter. But can you then ask questions about the text? That's unquestionably AI.
To me they're the same thing. If there's a bunch of training data that is fed into a system that creates a model, then it's not traditional programming, where someone laboriously writes out if statements by hand. AI and ML aren't, as far as I'm aware, rigorously specifically defined terms. They're words that marketing picked up and ran with it. To me, what matters is: is there a black box somewhere in the system that's a bag of numbers, or is it code that a human could dig in and read.
There are both ML that is not AI, and AI that is not ML.
For example, if you pick them manually, decision trees can be AI but not ML. Video game character behavior is a trivial example.
Eliza for example is also not ML, but could be called AI.
Likewise, there is ML that is not AI. Such is debatable, because you could always argue that using machine-learning on anything results in intelligence. The way I see it, things like image enhancement or voice replacement are not artificial intelligence at all. I probably could not define a hard line where it becomes artificial intelligence though.
>Video game character behavior is a trivial example
That makes no sense. "Manually picking" items from a decision tree is literally not "AI".
I don’t know what the op meant by manually picking but the expert systems of the 80s and case based reasoning systems of the 90s used fairly static decision functions and were explicitly called AI at the time.
AI and ML have very clear definitions[1]. ML has been a subset of AI, always has been. Latest marketing or "scare quotes" doesn't/shouldn't change that. Especially not in a technical forum like HN.
[1]: https://en.wikipedia.org/wiki/Artificial_intelligence
Ctrl-F Machine Learning. Apple Watch alerts are Machine Learning
It’s machine learning, which people routinely called AI not so long ago.
ML was always marketed separately as AI/ML, with AI being things like CNNs/RNNs/BERTs and such. Always felt like a distinction without a difference.
Laypeople are changing how “AI” is used in common language, like they previously did for “algorithm” and “crypto”.
The textbook definition of AI is a system that solves problems that are difficult for humans. Whether the approach uses formal logic, machine learning, neural networks as a special case of machine learning, optimization, search problems, etc. does not matter.
I don't think so. ML was always associated with AI. When it wasn't, it was called statistics.
I never heard people calling machine learning "AI" until large language models made it trivial to market it as such. Like, I remember back when Netflix, for instance, was going around advertising how machine learning (not AI) powers their recommendations.
> I never heard…
You should listen better. The University of Edinburgh had an entire Department of Artificial Intelligence when I was an undergrad there in the 1990s, and one of the things it researched was machine learning.
I don't see how including machine learning under the artificial intelligence umbrella counts as calling machine learning AI.
My local supermarket places the almond milk in the dairy section, and some people find this very upsetting.
My local cvs refused to let me buy non-alcoholic Bloody Mary mix aka spicy tomato juice without ID, because it was slotted in the alcoholic category.
That example kind of illustrates my point though? ML being in an AI book doesn't mean ML is AI, just as being in the dairy section does not make almond milk dairy.
It illustrates that some people can’t distinguish between a useful label by association for the general public, and their own desperate compulsion to litigate hair-splitting category distinctions.
Someone who was truly on the ball on this matter might’ve observed that Edinburgh in the 90s was so balkanized by internecine personality conflicts that most research that might later be strictly labelled “machine learning” actually took place in adjacent units and not directly under the DAIry. But I suppose you haven’t heard that, either.