Who Aligns the Aligners?
prestonbyrne.com62 points by timr 4 hours ago
62 points by timr 4 hours ago
> Nobody knows the answer and history is no guide, save that apocalyptic predictions about new technologies have, to date, all been wrong.
If any past apocalyptic prediction had been true, you would not have been around to write this sentence and we would not have been around to read it. It’s not a very convincing argument when we cannot in principle observe the counterfactual.
But warnings (e.g. not apocalyptic but damaging or having any effects) have come true, by and large; people warned about the effects of pollution / co2 emissions / global warming over 150 years ago [0], to name a popular example.
I don't think there are many truly human created apocalyptic events out there besides global thermonuclear war, and I don't think the worst-case AI superintelligent killer robot scenario would be apocalyptic. Unless it somehow triggers global thermonuclear war, but then it becomes a chicken-or-egg question (and it'll probably be humans pushing the proverbial button, as nukes are '60s tech and (hopefully) airgapped etc).
I suspect they have not all been wrong for some people, e.g. native peoples at various times prophesied I’m sure about various technological marvels their more advanced colonizers/enemies/etc.
By that logic, literally any ridiculous supposition must be taken seriously, because you can’t prove a negative.
This is the civilization-scale Quantum Immortality argument... no civilization can be around that witnessed the end of civilization.
(I refer to it that way rather than a "variant" because Quantum Immortality should always be parameterized by an acceptance function... for any given acceptance function, that function will be preserved in the quantum multiverse, and scoping the precision of the function affects the percentage of universes that retain it rather than whether it will be retained. There's actually nothing in that definition that requires that the acceptance function describe a "conscious system", you can stick something that accepts a "civilization" in there just as easily. Or "this rock over here". Or any other thing you like. Conscious systems are just a particularly interesting case to us conscious systems.)
It's not a vacuous argument, though it is a funky one with very odd characteristics. But if you accept the suppositions underneath it, it does follow and is not vacuous. The suppositions are certainly not completely obviously true on their own terms, though.
>Quantum Immortality argument
I never really understood what is the explanation for ... old people that we notice as dying. In today's times you might have an argument that some kind of life extension could pop up and some of the people alive today could have that as an excuse.
But like 100 years ago? How would people dying of old age "continue living"?
From the point of view of the old people, exponentially large injections of improbability.
This is also where it is important to understand the argument is parameterized. If your acceptance function is "a full human body living a full human life in the way I'm used to", then you're asking for a huge whackload of improbability to keep that body alive. If, on the other hand, you merely ask for "the minimum system that stays conscious", while that is still a huge whackload it is exponentially smaller, by a very substantial factor.
Anyone who takes any sort of comfort in the argument doesn't understand it. It preserves minimal consciousness, for any and all definitions of "minimal consciousness". It does also preserve bodies, but at an exponentially smaller probability. You can even use as your acceptance function "civilization exactly as we know it today, with everyone currently living still alive in 1000 years" and that comes at a yet smaller exponential probability. All of these "exponential probabilities" as separated by numbers of zeros that themselves require exponential notation to write. But the universe itself is making so many splits and branches all the time that it has the "budget" for these probabilities to be true, somewhere in the quantum multiverse.
Or, put another way, any event that you can draw a non-zero probability pathway for occurs. It is not impossible for me to spontaneously have my particular genetic defect reversed in every cell in my body, so there are branches where that happens... they're just exponentially unlikely compared to what we actually experience. It can literally be in %10^(-10^1,000,000,000) of the universes... but if it's not zero, it will happen, if the quantum multiverse is the correct view.
Quantum immortality is just an absurd quasi-religious pretense of physics that caters to those that fear death but feel embarrassed to participate in the traditional socially approved coping mechanisms.
I do not think death exists as a thing. We're shoving a lot of stuff into the concept. Pain, loss of information, of the one who dies. Fear of the unknown, what happens next. Death is a very volatile concept if you really try to pin it down. When you take all the layers off...you are left with nothing really.
But yes, the apparent loss of information seems to be true, for us at this technological level.
I mean sure, in the end you can have some heaven, some benevolent alien species reaching back into some field and copying all of your material structure data right at time of death and rebuilding you in some kind of resurection center. The space of what is possible is wildly vast, we know or can think of very little by comparison. If you give up some day to day coherence then you can almost always find something that would work and make sense once there.
It could even be us, future us, being able to somehow recover past data and literally reassemble all of the humans that ever lived. They aren't as many as you'd think, in the grand scheme of things.
I’m not sure what logic you have imputed to my comment, but I’m having trouble connecting what you wrote to anything I was saying. I am criticising the argument “every past apocalyptic prediction was wrong, therefore this future apocalyptic prediction is wrong” (specifically, that it carries exactly 0 weight, because the counterfactual world where at least one past apocalyptic prediction was true, is also a world that does not contain this argument - so there is no world where the argument’s precondition can, even in principle, be false. Arguments are a little bit like rocketships in that they need “reaction mass” - there has to be some amount of false for the truth to push against in order to move our opinion.)
> I am criticising the argument “every past apocalyptic prediction was wrong, therefore this future apocalyptic prediction is wrong”
That is not the argument being made. That is the straw man version you are presenting. Read the full paragraph, and the one after, and it's obvious:
> There are myriad policy responses. Whatever we do, those responses will require popular consent and careful deliberation. No one person, or one company, or one movement, knows the answer and history is no guide, save that apocalyptic predictions about new technologies have, to date, all been wrong.
> History does provide a great deal of guidance, however, about the use and misuse of government power. It tells us that the state is in fact likely the worst possible custodian for the most powerful publication and data analysis technologies.
Emphasis mine.
No, that does not follow.
Any other argument against <ridiculous apocalyptic supposition> is perfectly fine, just not the "it hasn't happened before" one.
You're arguing in favor of a teapot circling Mars. We can't prove it's not there, but telling me that "nobody has ever found it" is an invalid argument is obviously absurd.
In that quote, the author is telling you that all of your past predictions of teapots have been wrong (which is 100% true -- the ridiculous doom has been going on since at least GPT-2. remember how that was "too dangerous to release"?) Show me why I should believe in this one.
This is a grevious misuse of Russell's Teapot. You realise your assertion, if followed to its logical conclusion, is that because civilization has existed for this long, it can never collapse no matter what.
> In that quote, the author is telling you that all of your past predictions of teapots have been wrong
Past predictions have zero influence on future ones. If you flip a coin 10 times and get heads every time, the probability of your next coin flip coming up heads is 50%, not 100%, because it is independent of any prior trials.
Moreover, if Alice and Bob each make 100 predictions and are 100% wrong, but Charles makes 10 predictions and is 100% correct, Alice and Bob's predictions have zero bearing on whether you should trust Charles' predictions. They are not even a factor to be considered.
I don't think I even disagree with you in the broad picture. I am not concerned about language models. But you have committed three blatant logical fallacies in a row. You literally could not make a worse case for your position if you tried, I suspect.
You're making a ridiculously improbable prediction that can never be falsified -- akin to beliefs in a vengeful God. Russell's Teapot is exactly the correct metaphor.
The philosophical burden of proof lies on the person making empirically unfalsifiable claims. That is the point of Russell's Teapot.
> The philosophical burden of proof lies on the person making empirically unfalsifiable claims.
You appear not to understand what a prediction is. "I believe X will likely happen because Y" is different from an assertion of fact that "X will happen". A prediction cannot even be empirically proved! You are simply suggesting that nobody is allowed to make a prediction, ever. Russell's Teapot, too, is not an argument that you cannot believe in God. It is an argument that the unfalsifiability of a claim is not a basis to assert that God exists.
You can make whatever predictions you like. Don't expect other people to believe them and act on them unless you provide evidence.
This is Pascal's Wager, which kind of brings us full circle again to AI doom predictions.
Apocalyptic predictions normally get everyone into gear to prevent the threat. Not a great argument for "let's see what happens if we just sit around this time"
I'm not even sure that the author is correct. The industrial revolution was pretty apocalyptic for the people forced to work in dangerous factories for minimal pay.
Hell, maybe the Silurian hypothesis is true and past predictions were correct.
I always struggle to understand such an anthropic principle type of reasoning.
>If any past apocalyptic prediction had been true, you would not have been around to write this sentence and we would not have been around to read it
So what? How is this [proportion of possible universes in which we can read the failed prediction] logically relevant to our universe?
It just doesn't click for me, it smells like some additional premise is smuggled, possibly the same as in the simulation hypothesis argument.
Plus, plenty of negative predictions turned out to be entirely true. And then it took a lot of time and effort to counteract them.
It is not like history was composed of happiness and roses only. Some of the really really bad stuff happened within last 100 years and a lot of that bad stuff was enabled and made possible by technology.
Such as?
Like it's fine to say this but the worst exterminations of human beings weren't enabled by anything much kre sophisticated then the pointy stick, or at most, the railroad.
We're in the middle of finding it out with climate change.
Middle?
We're just starting. The air conditioning technology didn't hit its wall of fire yet. What we're going to do when that tech will cease to cool us?
Every AC system (from refrigerators to air conditioners) has an upper limit, which is around 41 degrees Celsius for T1 class.
> What we're going to do when that tech will cease to cool us?
Move. This is already happening, with water shortages forcing people to move. Climate change will (likely) cause famines and many deaths, but the bigger effect will be mass migration.
> Move.
Another planet? How?
We assume that we will have a planet to live at the end of the day. I'm not so sure about it for the long run.
You can't AC your way out of climate change anyway: they make heat. In sparsely populated areas that might be okay, but in density, you're never far from the hot exhaust of an AC unit; they exacerbate the problem.
Why would 41C be the limit for the hot side? How does it work now in hotter places?
In short: Things degrade.
More: https://angliancompressors.com/blog/how-ambient-temperature-...
There's a T3 class for up to 52 degrees Celsius. While it's higher than 41 degrees, it's not significantly higher:
https://zerohvacr.com/blogs/t1-t2-t3-explained-climate-class...
Is 52C a physical limit?
Yes. The compressor can't pump more heat outside since it's already hot from working and trying to pump out the extracted heat as well.
For T1 class of devices, that 41 degrees is also a physical limit, not an artificial cut, because of their design.
You know, thermodynamics.