OpenTPU – An open-source AI accelerator, developed by AI
github.com333 points by fsbonetto a day ago
333 points by fsbonetto a day ago
Really dumb question from a software guy. Why aren't the labs burning their frontier models into chips already? Seems like the performance gains and cost per request would be worth it. That said, I understand neither the economics nor the physical challenges to doing this.
Model SOTA moves faster than chips can be designed or produced. You'd need to commit to a particular model for years to get payoff while still burning buckets of money producing new SOTA models to keep up with the competition.
It's why everyone and their dog runs these things on GPUs. When a new model supercedes the previous one, so long as you've got the memory for it your chips aren't obsolete.
I'm looking forward to someone picking a model to be "good enough" (say, qwen 4.0 or something) and selling them as peripheral hardware
At this point, LLM's are "good enough" for all kinds of tasks. Instead of making them more capable, now the efforts are making them smaller and cheaper.
All aboard! We're racing to the bottom now.
IMO this is the dream scenario! Cheaper and faster at the current level of capability gives us incredibly useful tools without the worst of the risks people fear. (Though there are certainly already great risks at the current level of capability as well.)
Racing to the bottom is literally the outcome I am most afraid of
I don't want to live at the bottom
Say more. Why would cheap inference be bad?
Terrible for me because I have to compete with AI for jobs
Ah that makes sense. I'm skeptical that AIs doing things on their own will be competitive with humans using AIs to do things. But I'm not incredibly confident in this and I think it's sensible to think the opposite. But to me, I just think about all the things I can accomplish with the aid of very good, fast, and cheap inference.
I think the sad part is that whether it's any good or not, it will be cheaper, and that will be enough even at current performance levels to cut a huge number of jobs. I think people exist in the present moment largely because legal questions about liability are not settled well enough for executives to start the great purge. I believe on some level my continued employment is simply because people in leadership positions would be incredibly unwise to not have a human to blame when things start going wrong. People are expensive insurance policies for the time being. Once there is some legal framework to absolve executives of their own culpability or they figure out a good way to insulate themselves from personal responsibility, then the real "fun" will begin.
The problem comes in to how many bullshit jobs we'll have to create to ensure that enough people can still earn a living to ensure that we don't have riots and revolution in the street.
Companies being profit seeking entities that are actively hostile to social wellbeing would gladly put all of our money in a machine money making loop and leave all but a few humans out.
Like mostly all tech, AI is destined to get cheaper with time. We're still pretty early on and are experiencing a hardware crunch, which will ease eventually.
AI doesn't have workers rights, doesn't burn out, doesn't get sick, can be instantly onboarded, etc. The second we can be fully replaced with AI, we will be. Plan accordingly. I'm pursuing FIRE and considering moving into a trade.
> The second we can be fully replaced with AI, we will be. Plan accordingly. I'm pursuing FIRE and considering moving into a trade.
Doesn't matter what plans you make, the impact will be across everyone!
Moving into a trade won't help, because the supply is doubling while the demand is lowering. Fewer people with money to spend; they'll fix their own damn toilets if the decision comes down to "buy food" or "hire plumber".