South Korea to spend $1T on more memory chip production and humanoid robots
arstechnica.com221 points by jnord 10 hours ago
221 points by jnord 10 hours ago
The title sounds to me like: I am going to spend $1000 in groceries and dance lessons. That is, two very different things lumped together.
Memory chips are like groceries, essential commodity parts, a no-nonsense investment. Humanoid robots are like dance lessons, it is cool, it is sexy, and it may pay off in the future, but the value is much less certain.
$585B on new fabs, $357B on AI data centers, and $5.8B on humanoid robots. One of those numbers is not like the others
I don't have the historic numbers at hand, but I would assume that for each of those categories this is a similar proportional increase, so it's similarly notable to mention the increase in spending on humanoid robots.
Androids (humanoid robots) will require loads of ram, and loads of model training under the current paradigm. So it sort of makes sense. At least, I see robots as the top of the pyramid.
Autonomous (non-teleoperated) humanoid robots that can do useful work in an unfamiliar environment do not exist. And nobody's close enough to making them to understand if they're possible with our current level of technology, let alone how.
Most initial work for them would be in familiar, well-controlled environments - replacing humans in existing factories. I think whether they'd be cost effective for that will remain unknown even after a few years in service though.
We’re experiencing gpt-2 moment in robotics now. This means in about 2-3 years they will do useful work (cooking, repairs, cleaning, etc).
We said the same thing about Waymo, that it was perpetually in the future. It took them less than a decade. The robots today are functionally capable, they don’t have the right fuzzy intelligence yet. It’s purely a data problem (lack of) and a lot of people are working on it.
Are you saying autonomous driving is a solved problem, even at scale? I haven't seen any Waymo in my small town in Southern Europe yet.
It's not just a data problem, it's a hardware problem. Transformer-based robots require even more processing power than plain LLMs, as they also need to process visual and spatial/touch input. We don't have GPUs capable of fast inference on a SOTA LLM that would fit in a robot brain form factor, let alone also run fast enough spatial and visual processing. And there's currently nothing even approaching a feasible solution for cooling such a device.
If there's no unknown unknowns in the brain, it's most likely possible. As the universal approximation theorem and empirical results of scaling SGD+RL suggest. Whether it will be economically viable remains to be seen. The human cerebellum has a peculiar structure and 80% of the brain's neurons after all.
Real neurons are orders of magnitude more complex than their artificial pseudo-approximation (it is all based on the century-old understanding of how neurons work). You can think of _individual_ biological neuron as an analog of the small artificial neural network. You can see this simple visual explanation on YouTube[1]. So we aren't even close. It doesn't mean the AI is impossible, it just means people underestimate the "computing power" of real brains, as well as that AI, even the future one might be totally different in how it works from the natural intelligence.
But for biological neurons to do something that can't be efficiently approximated on a digital computer (but conductive to useful information processing) they need to have unknown unknowns (well, partial unknowns like an unknown quantum algorithm will do too).
We don't know the violations of the physical Church-Turing thesis that are conductive for machine learning. We don't have evidence for their existence in the brain (although, the brain would be the prime candidate for finding them as evolution works directly with the true physical laws).
BTW, large ANNs don't try to model how the brain does things. They are trying to mimic what the brain does. So, using "how many transistors/artificial neurons it takes to model a biological neuron" is not a good approach.
We have no evidence. We even have no solid theories how this can work (Penrose's OrchOR is "OrchOR somehow taps into mathematical knowledge somehow encoded into the structure of spacetime"). But people, for some reason, insist that there should be something there. I can't attribute it to anything else but to deeply entrenched feeling of human exceptionalism.
You're talking about a philosophical debate whether the brain is computable, the other commenters are pointing out that even conservative estimates point to a brain-like NN requiring over a quadrillion parameters.
...assuming that modelling the physical structure of the brain is the only way to model its functions.