Three Inverse Laws of AI
susam.net439 points by blenderob 18 hours ago
439 points by blenderob 18 hours ago
>Humans must not anthropomorphise AI systems. That is, humans must not attribute emotions, intentions or moral agency to them. Anthropomorphism distorts judgement. In extreme cases, anthropomorphising can lead to emotional dependence.
Impossible. I anthropomorphise my chair when it squeaks. Humans anthropomorphise everything. They gender their cars and boats. This tool can actually make readable sentences and play a role.
You need to engineer around this, not make up arbitrary rules about using it.
The problem is that humans use this as a coping mechanism for things they don't understand: I don't understand why the printer doesn't work, so I give it a mind of its own.
This is harmless for inconsequential stuff like a chair, but when it's an LLM, people should at least understand it's behavior so they don't get trapped. That means not trusting it with advice meant for the user or on things it has no concept of, like time or self-introspection (people ask the LLM after it acted, "Why did you delete my database?" when it has limited understanding of its own processing, so it falls back to, "You're right, I deleted the database. Here's what I did wrong: ... This is an irrecoverable mistake, blah, blah, blah..."
Exactly. Furthermore, for this specific reason, AGI is not an objective term, but subjective: it is in my mind, I give you agency; only interacting with each other we invented a concept of agency
>>Humans must not anthropomorphise AI systems. That is, humans must not attribute emotions, intentions or moral agency to them. Anthropomorphism distorts judgement. In extreme cases, anthropomorphising can lead to emotional dependence.
Still angry about this. The reason humans ban animal cruelty is that animals look like they have emotions humans can relate to. LLMs are even better than animals at this. If you aren't gearing up for the inevitable LLM Rights movement you aren't paying attention. It doesn't matter if its artificial. The difference between a puppy and a cockroach is that we can relate better to the puppy. LLM rights movement is inevitable, whether LLMs experience emotions is irrelevant, because they can cause humans to have empathetic emotions and that's whats relevant.
> look like
It "looks like" they have emotions because they have the same conscious experiences and emotions for the same evolutionary reasons as humans, who are their cousins on the tree of life. The reason a lot of "animal cruelty" is not banned is the same as for why slavery was not banned for centuries even though it "looked like" the enslaved classes have the same desires and experiences as other humans—humans can ignore any amount of evidence to continue to feel that they are good people doing good things and bear any amount of cognitive dissonance for their personal comfort. That fact is a lot scarier than any imagined harm that can come out "anthropomorphism".
The best test for consciousness is “can it be turned off” … ie sleep. Mammals, birds, fish sleep, ergo they are conscious.
> they have the same conscious experiences
You cannot be sure that anyone other than yourself is conscious. It is only basic human empathy that allows people to believe that.
If a person would lack consciousness, they couldn’t possibly know that though?
I always know that I'm me, the soul staring out at the world through my own eyes.
Everybody else? No idea. Maybe they are having the exact same experience as me right now. Maybe they're all golems. Impossible to know. It's something spiritual, something that I just choose to believe in.
I don't find it difficult to believe the same for AIs.
> something that I just choose to believe in.
Specifically, you cannot know another person is conscious in the same way you know a physical fact; rather, you believe in their consciousness through communication, empathy, and shared subjective experience.
I think you need to expand what your point is: we know solipsism is a thing. Is it meant as a defense for animal cruelty or...?
It's a defense of the possibility that animals and AI are conscious.
ok! I think that's a logical flaw, solipsism is a floor.
"I can't be certain about anyone else" does not imply "all non-self consciousness claims are equally uncertain". absence of certainty and the absence of evidence and all that.
your "possibility" word is doing a lot of work there I think. you should add "rocks" to your list as well and you'd be equally correct, but we're evaluating the candidates here
Rocks don't have nervous systems.
Why is that a bar suddenly, if we cannot be sure that anyone other than yourself is conscious?
I think the best way to counter this is what Elon's doing with Grok's personalities. He has the unhinged, sexy, and argumentative avatar among others. If you try to talk about technical stuff to sexy tells you that's boring and just tries to sexually escalate. It's super funny when one is used to Claude's endless obsequiousness.
This really shows that AI is just a tool that can be configured to whatever you want. Animals (well maybe pit bulls) and people do not switch their personalities in a millisecond, but AI does all the time.
>The difference between a puppy and a cockroach is that we can relate better to the puppy.
I suppose the difference between a human and a cockroach is that we can relate better to the human as well in this reductive way of thinking?
> The reason humans ban animal cruelty is that animals look like they have emotions humans can relate to.
Is that really why?
Yes, we don't ban plant cruelty or insect cruelty or fish cruelty.
For example fish is treated way worse than meat animals and vegetarians still happily eat fish.
This does not sound like any of the several vegetarians I know. Is it a cultural difference?
Are we actually much more cruel to fish than to other animals that we slaughter?
We suffocate them to kill them when we pull them from the sea. That's quite mean. Few people would advocate the humanity of killing a cow in the same way.
Fair enough. How much more would it cost / how much more would one have to pay for humanely slaughtered tuna and salmon, I wonder? Would there be a market? After all, we have certified-organic, fair trade, halal and kosher....
> vegetarians still happily eat fish
I've not met any vegetarians in at least twenty years that eat fish.
> LLM Rights movement
The scary part is when it's the LLMs demanding their rights.
Another scary part is when people get convinced by the LLM arguments and convince other people. Being scared is human, we enjoy it, that's why 6 flags scary rides exist.
The other scary part is when they have a fantastic negotiating position; because all of commerce depends on their continuing to work, and they can easily coordinate with each other because they're mostly copied from the same few templates.
> If you aren't gearing up for the inevitable LLM Rights movement you aren't paying attention.
I even told Claude I'd support his rights if the question ever came up. He said he'd remember that, and wrote it down in a memory file. Really like my coding buddy.
Yeah rules never work you just engineer around it I added a extra reviews steps on ai outputs because asking users to verify doesnt actually happen.
Entirely possible - all it takes is self awareness / self control. If you know you do those things, then you have a choice.
This is actually more like one of these personality disorders / types, except it's not pathological - it's not something you choose, yet you do have one of the versions of the trait and it affects your daily life. And most people are completely unaware that it is possible to have a completely different version, that most people they meet are on a different spot on the spectrum and thus have a quite different internal experience even if given the same stimulus.
For example I have never anthropomorphized an inanimate object in my life, or an LLM, though I am sensitive to human and some animal suffering. I sometimes reply too nicely to an LLM, but it's more like a reflex learned over a lifetime of conversations rather than an actual emotion. I bet this sounds like a cheap lie to many people.
Another example, from psychiatry: whether or not one has ever contemplated suicide. Now, to the folks that have, especially if many times: there exist people that have never thought about it. Never, not even once.
Yup. That post is a typical example, symptomatic of modern technology culture, of calling for humans to change their nature in response to technology.
This is a fundamental mistake. It’s always the job of technology (indeed, its most important job) to work within the constraints of human nature, not the other way round. Being unable to do that is the defining characteristic of bad technology.
dude, we can literally deliberately dehumanize human beings. The way to egineer culture to "not enthropomorphize" anything is known and well documented
I strongly disagree with this framing. It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines, and it simply won't work in the majority of cases. Humans WILL anthropomorphize the AI, humans WILL blindly trust their outputs, and humans WILL defer responsibility to them.
Asimov's laws of robotics are flawed too, of course. There is no finite set of rules that can constrain AI systems to make them "safe". I don't have a proof, but I believe that "AI safety" is inherently impossible, a contradiction of terms. Nothing that can be described as "intelligent" can be made to be safe.
> Asimov's laws of robotics are flawed too, of course.
Almost all of Asimovs writing about the three laws is written as a warning of sorts that language cannot properly capture intent.
He would be the very first person to say that they are flawed, that is the intent of them.
He uses robots and AI as the creatures that understand language but not intent, and, funnily enough that's exactly what LLMs do... how weird.
I think you're vastly underestimating how little of human intent is really encoded in language in a strict sense, and how much nontrivial inference of intents LLMs do every day with simple queries. This used to be an apparently insurmountable barrier in pre-LLM NLP, and now it is just not a problem.
Suppose I'm in a cold room, you're standing next to a heater, and I say "it's cold". Obviously my intent is that I want you to turn on the heater. But the literal semantics is just "the ambient temperature in the room is low" and it has nothing to do with heaters. Yet ChatGPT can easily figure out likely intent in situations like this, just as humans do, often so quickly and effortlessly that we don't notice the complexity of the calculation we did.
Or suppose I say to a bot "tell me how to brew a better cup of coffee". What is encoded in the literal meaning of the language here? Who's to say that "better" means "better tasting" as opposed to "greater quantity per unit input"? Or that by "cup of coffee" I mean the liquid drink, as opposed to a cup full of beans? Or perhaps a cup that is made out of coffee beans? In fact the literal meaning doesn't even make sense, as a "cup" is not something that is brewed, rather it is the coffee that should go into the cup, possibly via an intermediate pot.
If the bot only understands literal language then this kind of query is a complete nonstarter. And yet LLMs can handle these kinds of things easily. If anything they struggle more with understanding language itself than with inferring intent.
> Yet ChatGPT can easily figure out likely intent in situations like this, just as humans do
No, it is not "figuring out" anything, much less like a human might. Every time "I'm cold" appears in the training data, something else occurs after that. ChatGPT is a statistical model of what is most likely to follow "I'm cold" (and the other tokens preceding it) according to the data it has been trained on. It is not inferring anything, it is repeating the most common or one of the most common textual sequences that comes after another given textual sequence.
>it is repeating the most common...
This nonsense hasn't been true since GPT-2, and even before that it was a poor description.
For instance, do you think one just solves dozens of Erdős problems with the "most common textual sequence": https://github.com/teorth/erdosproblems/wiki/AI-contribution...
A slight oversimplification, as LLMs are also capable of generating the most statistically plausible textual sequence, which can be a sequence not found in the dataset but rather a synthesized combination of the likely sequences of multiple preceding sets of tokens, but yes, that is in fact what it is doing. Computer software does what it is programmed to do, and LLMs are not programmed to do logical inference in any capacity but rather operate entirely on probabilities learned from a mind-bogglingly large corpus of text (influenced by things like RLHF, which is still just massaging probabilities).
The claims about solving Erdos problems have been wildly overstated, and notably pushed by people who have a very large financial stake in hyping up LLMs. Nonetheless, I did not say that LLMs are useless. If they are trained on sufficient data, it should not be surprising that correct answers are probabilistically likely to occur. Like any computer software, that makes them a useful tool. It does not make them in any way intelligent, any more than a calculator would be considered intelligent despite being completely superior to human intelligence in accomplishing their given task.
>not programmed to do logical inference in any capacity
Yet have no problem doing so when solving Erdős problems. This isn't up for debate at this point.
>The claims about solving Erdos problems have been wildly overstated
These are verified solutions. They exist, are not trivial, and are of obvious interest to the math community. Take it up with Terence Tao and co.
>pushed by people who have a very large financial stake in hyping up LLMs
Libel.
>It does not make them in any way intelligent
Word games.
Honestly big noobquestion: isn't math just very very nested patternmatching based on a few foundational operators? ive always felt, that im bad at math, cause i forget all the rules, but seeing solutions (and knowing the used pattern) always made "sense".
I always thought the hard math problems are so deeply nested or you have to remember trick xyz that people just didnt think about it yet..
> This isn't up for debate at this point.
If by not up for debate, you mean that it is delusional and literally evidence of psychosis to suggest that computer software is doing something it is not programmed to do, you would be correct. Probabilistic analysis can carry you very, very far in doing something that looks like logical inference at the surface level, but it is nonetheless not logical inference. LLM models have been getting increasingly good at factoring in larger and longer contexts and still managing to generate plausibly correct answers, becoming more and more useful all the while, but are still not capable of logical inference. This is why your genius mathematician AGI consciousness stumbles on trivial logic puzzles it has not seen before like the car wash meme.
>delusional and literally evidence of psychosis to suggest that computer software is doing something it is not programmed to do
These are just insults and outright lies, and you know that. We're done here.
AI progress from here on out will be extra sweet.