Are we offloading too much of our thinking to AI?
artfish.ai521 points by yenniejun111 3 days ago
521 points by yenniejun111 3 days ago
I don't know if this is a good framing. "Too much" is subjective, and every heavy AI user will assert that they're just unlocking their potential, that calculators didn't make us dumber, etc.
But to latch onto the calculator argument: if you outsource adding numbers to a calculator, you're still you. On the flip side, if you use an LLM do most of your thinking, what's left? We have people here who use LLMs to raise their children, to manage relationships, to design products. So what's your unique contribution to this world - is it the prompt you once wrote? You're standing in front of a token-generating machine, pulling a lever, sometimes receiving gifts. Is that your edge, your unique experience, your purpose in life?
Many LLM maximalists say they use the tech to learn new things, but to what effect? Are you going to apply that knowledge of physics or computer science yourself, or will you just prompt the LLM again?
In my mind, it's pretty simple: I'm a human, LLMs are not. If a human writes a novel, it's inherently worth more because it's hard-earned and anchored to experiences we share. I want to support that. And I want to be a human who can write novels, the old-fashioned way. I'm not good at lifting weights or running, so my thinking is the only thing I have.
Have you read the "Whispering earring" essay? I love it for the LLM era.[1]
You can treat AI as a whispering earring - "What should we do now? How do we fix this? What do you think?" Or you can treat it like an exoskelton - "Implement kd-tree with metric space xyz for this problem, mapping this to that blah blah".
That's pre-thought execution automation that makes review much simpler - you already know the shape of the desired output. The whispering earring is atrophy.
It cracks me up that you bring up the exoskeleton metaphor, because I'm pretty sure it originated from a 100% AI-generated essay that made it to the top of HN a while back. So I guess, AI is whispering things into our ears whether we notice or not.
Exoskeleton is such an obvious metaphor for AI (at least, one potential ways of using it) that surely no one, human or AI, could possibly claim credit for it.
I had been explaining it to others as 'My Iron Man suit' before reading the exoskeleton essay, months later. So, you are definitely correct.
It feels more like a BMX bike to me. The quality of what I get out seems to be a direct function of what I put in. The threshold of what I can achieve is raised slightly, the speed with which I can achieve it is raised quite a lot - but it's not a magic rocket ship that whisks me to far off hostile environments and protects me while I'm there. I'm still "naked". The code is still mine, and I can get off the bike at any point and walk.
Thats... not how most of the world is using llms. Students are getting measurably dumber (as in lacking actual topic understanding, since actually learning sonething properly is much harder than shallow knowledge), every other day I read about somebody losing their wealth due to listening to llms with finances.
Other recent articles - psychologists start seeing people losing their uniqueness and many people slowly veering towards same middle bland opinions.
I think top parent is correct - we are in the process of visibly losing a lot from our humanity, much more than we are gaining. We shouldnt be listening too much to loudest mouths here that praise efficiency gains, many of those are 'semi-autistic' folks with tunnel vision focused on 1 aspect and ignoring larger picture.
I dont see any change of course possible, the sweet nectar of efficiency and offloading decisions and work is too strong a drug to ignore. But everything has its set of benefits and drawbacks, and drawbacks here seem pretty brutal so far.
I think the first usage of this metaphor in its modern meaning belongs to Norbert Wiener
>So I guess, AI is whispering things into our ears whether we notice or not.
Humanity is a weak hive mind species. You can, with effort, keep it from whispering it into your ears, if you really want that. But it's impossible to keep it from whispering in another ear attached to your hivemind. And the only difference in those ears is the propagation delay before it's rattling around in your own skull.
That's a pretty mid-tier metaphor on my part, along with Iron Man + Jarvis, that I don't think I'd even want to claim credit for :)
I will claim the CNC analogy though - I sometimes feel like a modern machinist just walking between machines and listening for screaming metal.
> I'm pretty sure it originated from a 100% AI-generated essay
There's no such thing. You're probably thinking of "AI"-regurgitated essay.
If you think the AI generated that metaphor, you haven't been paying attention.
> I'm pretty sure it originated from a 100% AI-generated essay
There's no such thing. You're probably thinking of a 100% "AI"-/regurgitated/ essay.
Reminds me of Daemon by Daniel Suarez. I could plausibly see us heading there for sure https://www.goodreads.com/en/book/show/6665847-daemon
Something I never considered much is what happens when everyone else is using the Whispering Earring.
You may be more free and independent, but you may also be unable to compete as everyone else easily gains wealth and success. Natural selection doesn't particularly care about freedom of consciousness.
Bleak.
Eventually, that becomes interesting. Before we get to that stage, I think we pass through (almost) everyone following almost the exact same advice as each other.
Such people are extremely predictable.
You may have already noticed, before LLMs, cliché? Talking points that make a group identifiable? Words and phrases that act as applause lights or cognitive stop signs? (That last sentence itself being a pair of clichés that you can use to identify where I hang out online).
Anyway, point is, LLMs will give us a memetic monoculture before they turn us all into a world of correctly personalised Whispering Earring wearers. That makes them predictable, that makes them exploitable. It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves.
(Of course, the fact I've said this in writing means this is in the training data; in the general case this means the LLMs will know that and account for that, but I suspect comments like this won't shift the needle all that much compared to the aggregate output of 3 billion people reacting to short-form emotional manipulation A/B slop)
This is already underway.
My dog often gets misidentified as a restricted breed. This used to make apartment hunting difficult because, occasionally, the property manager would visually ID the dog breed as banned, I’d have to go to the vet and get paperwork, potentially gene testing, arguing she wasn't, it was a whole thing.
But, recently, the apartment I moved into had an online portal where I had to upload a photo and it would identify the breed to determine if it was approved.
I correctly assumed the portal was using an LLM for this purpose. I wrote a script which submitted different photos of my dog to the major LLM providers until it found a photo which all the LLMs would identify as the correct breed.
I simply submitted that photo and, as expected, passed with flying colors.
I was anticipating that it would have reversed course and flagged you. Blessings on you, and your four footed friend.
> It'll be like playing chess against someone you know is using a specific version of Stockfish: even though it would beat you if you tried to fight the system unaided, you can win by asking your own copy of the AI to go one step further ahead, and it will be accurate precisely because it's playing against itself and reacting to its own moves.
I don't believe this is how chess works, and I don't believe this is how Stockfish works, and I don't believe this is how AI works.
Stockfish isn't winning because it's playing a better sequence of programmed steps, and having access to "the next version of Stockfish" doesn't mean it can "guess the next move" and play against that.
I didn't say "the next version of Stockfish".
You have Stockfish version n, see board state s. I have Stockfish version n, see board state s. I want to know what you're about to do, so I put Stockfish into state s, ask it what the best move is, and I know you'll make that move because I know you'll ask Stockfish version n the same question of the same state. I now know board state s+1.
The steps are not pre-programmed, but the program itself is (modulo hardware imprecision) deterministic. If there's a RNG in there then sure, this doesn't work as easily as I wrote it; and there may be randomness in the thing that this is a metaphor for, regardless of if there's one in Stockfish or not, but that's not hard to work with when you want to win against an aggregate: we invented the field of statistics to deal with random numbers because they come up so often.
There is deliberate randomness in stockfish. The easiest way to see this is from the fact that, when playing the white pieces, it won't play the same opening move every time. Often it's e5, but it also goes for e4 or Nf3 or something else entirely.
This is by design, and very much necessary for a competitive chess engine. Otherwise, people could do basically what you say: Run an offline (as in, ahead of time, with ample compute resources) search against stockfish that finds a line where it loses, then make an engine that plays that every time.
As a consequence, even if you know that your opponent is running stockfish, you can't really use that against them. Your best bet is also just running stockfish.
Jacek Dukaj in his book "Perfect Imperfection" (Polish hard sci-fi about alternative physics, mind uploading and enclosed universes) wrote about entities vastly superior to human-level intelligence having an ability to simulate humans to such a degree of precision that they were able to determine what a given human would say or do based on his own internal experience and knowledge in real time. The superintelligent beings didn't scan his brain patterns, they just had multiple simulations of him and chose the one that would best match his actions based only on external cues like position of his hands, body language, speech patterns, etc.
A solution to avoid being read to such a degree that your free will effectively is destroyed was to introduce randomness as a program in your own mind processed in an equivalent of a "cloud VM" - random changes in your body posture, hand movements, weird phrases introduced while you spoke could confuse the internal models of you enough to avoid hyper-predictability.
That's really good, I'll have to read some of his work (Jacek Dukaj). It makes me think about the role of indeterminacy and unpredictability in physics. And reminds me of..who was it, Gurdjieff maybe, who taught that humans are like sleeping machines, and that constant conscious effort is necessary to "stay awake". Many of his exercises for students had the purpose of recognizing one's own habits, routine patterns of thinking and behaving, and breaking out of them.
The sci-fi story also reminds me of, I think it was related to Karl Popper's book about "objective knowledge". There was a thought experiment about Mozart, if a scientist could quantify everything external about the composer, like the brain, the body, the room he's in, his life experiences - is it possible to deterministically compose new works exactly as the real Mozart would. And if not, what is the difference between the perfectly simulated Mozart and the real one?
This "randomness" is related to the ancient Greek concept of clinamen.
> Clinamen, derived from clinare, to incline: the unpredictable swerve of atoms in the atomistic doctrine of Epicurus. This swerving, according to Lucretius, provides the "free will which living things throughout the world have".
Stockfish already does that. You run it as long as you want. When it's calculating evaluations for move k+1, it already knows what move k was, because it has that evaluation. It can explore that tree first, before exploring other trees to try to improve on move k.
Stockfish never settles down to an exact move, unless it sees a forced mate sequence. You could leave it running indefinitely, but the exponential blowup will soon grind you to a halt.
What new versions of Stockfish buy you are optimizations and better heuristics. An older version of Stockfish can still beat a new one, if given enough of a compute advantage.
Your whole strategy basically involves having more compute than the other guy so you can look one step further. And yeah in an AI world having more compute or some exclusive data seem like obvious ways to get a leg up.
Yes.
Have you met most people?
Most people won't bother, find this kind of thought process to be somewhere between nerdy and paranoid.
asimov's "profession" is a very relevant SF short: https://www.inf.ufpr.br/renato/profession.html
It is literally impossible for everyone to obtain wealth. Wealth isn't a number or physical circumstances. Wealth is having economic power over the people around you. Success is a little more nebulous, but when thought of as a relative of wealth it's similarly contextual.
The danger isn't everyone but you getting wealthy. The danger is that wealth tends towards concentration. And it tends to concentrate around people who are already wealthy. The danger is, bluntly, that things will get worse for all but a few and most people will be so caught up in a red queen's race that they can't see how to stop.
No. The world has absolutely gotten wealthier over the last few centuries.
Wealth is obviously not zero sum. Humanity is far wealthier today than before the industrial revolution, and the trend is still towards increasing wealth.
As Led Zeppelin once said, "you know sometimes words have two meanings". That definition of wealth is all well and good for books about how markets are magical fix-everything pixie dust. But it's not the definition of wealth that applies when used to describe a person. A wealthy person is not someone with a lot of things. That's a hoarder. A wealthy person is someone who can use their economic power to shape the world around them.
That definition of wealth is all well and good for books about how markets are magical fix-everything pixie dust
No, it actually means something. We have access to refrigeration, climate control, near-limitless computation, entertainment, transportation, knowledge, communication, and medicine. All of this stuff would BLOW THE MINDS of medieval kings. Even if you're living in a 1BR apartment, working at Walmart and struggling to pay rent, you enjoy many luxuries Charlemagne could scarcely dream of.
Go to the store, grab a pineapple off the shelf, take it home, and eat it. Then read about the great lengths [1] the wealthy people of the past went to in order to try to grow pineapples in Europe, and how obsessed the culture became with this fruit.
If you're in that one bedroom, struggling at Walmart, you probably don't have a dishwasher or a washing machine and dryer, and possibly not air conditioning or heating, or at least, your electricity bill is enough that you have to think twice before setting climate control to a comfortable level. Your social life probably isn't anywhere near what Charlemagne had. You have a smartphone and that has brought you news from afar. But that news is terrible! Local politics, things are fucked, climate change is out of control, geopolitics is a whole other shit show. Charlemagne didn't have to deal with Israel and Donald Trump. His world was very different. Sure, I have a fridge and a microwave and can use that to sate my hunger in mere minutes, a luxury inconceivable to Charlemagne, but all the trappings of modern society, with TikTok and Reels hasn't made us globally happier. Seeing the emotional damage from whichever online platform surfaces drone videos out of Ukraine today for me as a form of wealth is one perspective. We've lost the plot, or at least I have. We're solving the problems in front of our faces without asking if that's the right problem to be solved.
>A wealthy person is not someone with a lot of things.
No one made that claim.
By nearly any sane metric the world is wealthier. You can verify this on sites like Our World in Data.
We are healthier, have more leisure time, are less likely to die in natural disasters, etc.
It's willful ignorance to define wealth in a way that wouldn't show humanity as having gotten more wealthy. You're doing harm by overlooking massive gains we've made and understanding why we made them and instead redefining the common sense understanding of basic terms.
>> By nearly any sane metric the world is wealthier.
I would also point out people living in "poverty" in the US would be considered quite wealthy in other countries like Cuba or Haiti.
Everything is relative to how you define wealth and what you're comparing it to.
You are talking about access to resources and technologies, which is basically constantly rising, we couldn't stop that if we wanted to, so that's a given and more importantly orthogonal to the wealth that buys you goods and services.
The latter is absolutely zero sum, because of everybody has a billion, a billion isn't that much any more. E.g. human trafficking exists because some people are rich and others are poor, and the fact that even those poor people might have access to things an emperor 1000 years ago would have dreamed of doesn't change that, because the people who treat them like slaves have even more. It's all about the relative difference between people who are alive today, the absolute increase compared to people who lived N years ago is a red herring.
> ... human trafficking exists because some people are rich and others are poor ...
No, it doesn't. This is an absurd claim, and nobody intelligent should take it seriously. Human trafficking exists because some people choose to do evil things. There are plenty of relatively-rich people who are actively trying to stop the evil of human trafficking.
Furthermore, just being wealthy has not, historically, made you immune to being made a slave. For just one example, look at the story of Joseph in the Bible. He was his father's favorite son, the heir presumptive to fantastic wealth for his time (that "coat of many colors" his father gave him cost the equivalent of what a sports car would today), and yet his brothers who hated him sold him into slavery while his father wasn't around to protect him. Throughout history, cross-border slave raiding has been a thing, too: it didn't matter how much gold you had tucked under your bed if the Vikings (or whoever) caught you alone, or outnumbered and overpowered your hired guards: once they hauled you away and slapped chains on you, you no longer had access to your wealth, and your life depended on making your captors happy with your work.
To think that slavery, ancient or modern, is caused by relative wealth differentials reveals an abysmal ignorance of both history and human nature.
> To think that slavery, ancient or modern, is caused by relative wealth differentials reveals an abysmal ignorance of both history and human nature.
I claimed no such thing, you read way too much into your missing the point.
> if the Vikings (or whoever) caught you alone, or outnumbered and overpowered your hired guards
Wait, I thought it was just because "some people do evil things", nothing to do with relative power?
I'm deliberately giving an illustration that shows that relative power throughout most of history had to do with physical strength and weapons, rather than money.
And if you didn't mean that slavery is caused by relative wealth differentials, then saying "... the people who treat them like slaves have even more. It's all about the relative difference between people who are alive today ..." was a very poor way of making your point. So what was your point that I missed, then?