The memory shortage is causing a repricing of consumer electronics
davidoks.blog311 points by d0ks 17 hours ago
311 points by d0ks 17 hours ago
The headline here under-serves the article in my opinion: this is a fascinating, deep explanation of how the memory market works and why increased demand for HBM (used by big GPU racks) hurts the availability of wafers for DDR and LPDDR (used by laptops and phones).
This also put a lot into perspective:
> these memory makers have learned a very particular lesson from the unforgiving history [deep drops in demand] of their industry: always leave demand unmet
I can't blame them for keeping some reserve demand ready so they keep having customers over the years.
I wonder if this is actually true in the long-term though. If they were to flood the market with lots of high capacity memory, then I think our programs would start using more memory too. As a result we might end up needing more memory faster compared to if they keep demand unmet.
Just consider that a chat application today takes more memory than a full 3D game with thousands of users (including chat) + the operating system. If 128 GB of RAM were the norm then there's a chance we might expect to buy 128 GB of RAM too.
But I suppose it's really mostly a question of how many dollars we expect to spend on memory rather than the specific amounts.
I'm no expert analyst on this topic, but I'm worried this time might be different for them. China finally has the technological capability to challenge this time around and they will gobble up the unmet demand, allowing them to get more experience and more money to catch up.
The MacBook Pro on which I’m writing this piece needs memory that can keep up with a powerful processor running many programs at once: so it uses a standard called DDR, “double data rate,” which runs at a reasonably high voltage and offers high bandwidth. The processor on my iPhone is less powerful, so it needs less data at any given moment; but voltage matters enormously, since every milliwatt allocated to memory is drained from the battery. So smartphones use LPDDR, “low-power double data rate,” a variant of DDR engineered to operate at lower voltages.
Apple has been using LPDDR in the MacBooks since at least 2015. I remember it was one of the complaint of the 2016-2017 MacBook Pros. They were still using LPDDR3 because LPDDR4 wasn't ready for production yet (despite regular DDR4 being available). The 2018 MacBook Pro's finally switched to LPDDR4.
Maybe author is writing this from an Intel Mac!
Quite possible. He did say his “powerful” MacBook Pro CPU is faster than his iPhone.
I’m pretty sure even an iPhone 11 chip is more powerful than a 2019 MacBook Pro CPU in ST. An iPhone 15 is more powerful than the fastest 2019 MacBook Pro Intel CPU in MT.
I suppose he can be using a 2019 MacBook Pro or older and an iPhone 14 or older and compares only MT speeds.
What was most surprising about all this to me was this line:
> So modern DRAM manufacturing is an extraordinarily complex and expensive process. Building a single state-of-the-art DRAM fabrication facility, a “fab,” will cost you about $15 to $20 billion; acquiring all the necessary equipment, like lithography tools and etching machines, will cost you another few billion; and then it’ll take you a few years of producing substandard and defective memory chips before your yields start to look competitive.
Extraordinarily complex and expensive! And yet I look at all the money being shuffled around between Nvidia and Google and Microsoft and Amazon and Apple and can't help but think that this is a tiny amount in comparison to what they're moving around on the stock market buying shares in each other.
Apple in particular has $20B in its couch cushions and is very vertically integrated and hardware-focused. Apple silicon is currently made by TSMC, but it seems they'd be a prime candidate to spin up their own memory fab.
I suppose the biggest problem to current executives at each company is the "few years" until that investment yields results, in the short term it's better to pay through the nose and buy GPUs with HBM at any price.
> it seems they'd be a prime candidate to spin up their own memory fab.
While Apple et al certainly have the money to tilt up their own fab, they're savvy enough to understand the memory market's long history of constant boom/bust cycles. I still remember the huge DRAM shortage in late 80s forcing my startup at the time to delay launching our new product for a year.
People assume Apple cares about vertically integrating cost but they're actually focused on integrating margin. Apple has billions in cash on hand and when they think about what to do with it, a key metric is Return on Capital, especially the margin that capital will generate. Since a core metric public companies are judged on is blended margin, they are looking for ways their bags o' cash can be put to work generating revenue at margins that will pull their current average margin up vs down.
Averaged over time, mainstream memory devices are historically one of the worst margin areas of the semi market. It's super expensive to tilt up a fab on a new node but once you do, turning the crank faster to make a lot more chips isn't too hard because mainstream DRAM tends to be quite uniform. So when a fab on a new node and/or RAM generation first opens, the margins tend to be pretty great. But as the node matures and/or the RAM generation goes from 'new' to 'commodity', competition heats up as everyone gets better at making more faster. Then they're tempted to maximize revenue by cutting prices until their mature fab is at 101% utilization. And that eventually drives margins down until someone's selling near cost to sustain their low-price-enabling volume - with occasional dips below cost when they get stuck holding excess inventory. That's why cash-rich companies with high margins like Apple are delighted to buy DRAM built with Other People's Money. As long as the DRAM market is under competitive pressure, Apple gets to shop their huge orders around to get the absolute lowest price on RAM that was built with other investor's low margin dollars.
> I still remember the huge DRAM shortage in late 80s forcing my startup at the time to delay launching our new product for a year.
That one was caused by manipulation by politicians, not market forces. Micron started a price war with Japanese memory manufacturers, the Japanese cut prices to compete, Micron sued them for "dumping". The saga ended with the 1986 U.S.–Japan Semiconductor Agreement, which, among other things, created production controls that limited the total dram supply. The level was set based on then current demand, and due to the rapid growth of demand at the time it almost instantly caused a massive global supply deficit.
The agreement also caused the rise of the South Korean memory industry, because the Japanese companies offloaded their now surplus equipment for cheap.
> That one was caused by manipulation by politicians, not market forces.
Funny how you then go on to explain it was market _actors_ that drove the politicians to act, but that this is nevertheless totally not an example of market _forces_.
Market forces are usually meant in the spherical cow sense, not that the cow has politicians on their pockets.
Here it's useful to point out that free markets can't exist without an external force keeping them in check.
This has become like opec's best years. Only there's no cartel at least not openly.
But there's another key bottleneck. Even with all the money in the world, getting those machines that etch the RAM could be a multi year ration shop queue. And they're not making those companies every day!
> This has become like opec's best years
Yes, except no DRAM maker is taking this as a signal to go deep into debt to double or triple capacity. It takes a minimum of 3+ years to dramatically increase capacity and no one expects this inflated demand bubble to last that long. Because they've seen this cycle before, they'll bump next gen capacity up a bit more than planned, maybe 15-25% because the current windfall profits are enough to build a buffer to absorb the hit if that excess capacity comes online in the next DRAM demand crash.
In the meantime, they'll just apologize to everyone for the "market conditions beyond our control" while banking as much of these crazy profits as they possibly can while it lasts. But deep down they remember they were starving just yesterday and know they'll be starving again tomorrow.
I understand why everyone's pissed at the "evil DRAM makers" but I also remember boasting about how I scored some crazy cheap RAM sticks not so long ago. None of us were shedding tears when they were selling at a loss just to survive.
> Yes, except no DRAM maker is taking this as a signal to go deep...
Incorrect. CXMT is taking this as a signal to open new fabs, create new products, develop new silicon manufacturing techniques, and enter the consumer market worldwide (except USA).
CXMT is a company with heavy state backing and control that started its current trajectory years and years ago. It has infinity money and isn’t allowed to go bankrupt. CXMT is reading political signals (Beijing wants indigenous AI up and down the vertical) as much as market.
Isn’t that the point?
Everyone knows the commodity market (outside HBM) is going to be margin crushed within a few years… by CXMT, demand swings, and so on.
So nobody cares to fully meet the demand today. They are perfectly happy telling customers (those not willing to sign a 10 year contract paying up front at least) to go pound sand.
They don’t even need an OPEC like arrangement. They are effectively perfectly coordinated already in dismissing customers.
To reframe your great comment:
Is it fair to conclude that DRAM is basically a commodity that can be specified well enough by a set of parameters?
If so it won’t allow you to get any competitive advantage in your products and thus wouldn’t be a business you want to be in as Apple.
Yes, that's what I meant by "Mainstream DRAM". The basic parameters are specified by standards like JEDEC, so the most common types can be used in many systems thus reach high volumes. This can be less true for specialized RAM like HBM or the highest performance grades.
The only reason I can see Apple do this is if it enables them to sell entry level devices with vastly more ram than the competition can afford. Say entry level MacBook Pro with 256GiB ram to facilitate running frontier level local models. If that is an edge they want to have.
In what world would 256GB of RAM be “entry level”?
In the same world 8GB used to be unfathomably huge not that long ago?
Hell, an 8GB hard drive was unfathomable when I was a kid in the 90s. I remember getting a 30 megabyte drive for our Mac LC.
A world where Apple has invested in their own fabs, so they can sell devices with drastically more RAM than their competition at entry-level prices.
I'm no business expert and Apple is of course in a unique position, but owning your own fabs has rarely worked out long term. They require eye watering amounts of CAPEX that needs to be amortized over a timeframe that's longer than apple's products. Today's bleeding edge fabs become tomorrow's "cheap" fabs that pump out chips that don't need to be bleeding edge for the components that go into everyday products like microwaves, cars, etc.
One of the reasons Intel fell behind is that they couldn't give access to their competitors for business reasons, and therefore could never scale as high as TSMC could.
There are many other reasons, but accounting is a huge one. Unless there is a huge ROI or something else we don't otherwise know, I don't see Apple adding such expensive deprecating assets onto their books as chip fabs.
Why would they sell a device with 256GB of RAM as the lowest-spec device rather than making 8 32GB or 16 16GB machines as their entry-level?
Apple’s not exactly famous for their low pricing on spec upgrades nor competing based on being the price leader…
If 256GB of RAM enables them to run on-device AI models that (for reasons) are a key feature differentiator?
Personally, I think there's no way memory heavy inference moves on-device (vs cloud) due to the economics, but it's not impossible technology + platforms go that way for currently unforeseeable reasons.
Right. I’m not arguing that Apple wouldn’t offer a 256GB model if they could make money doing it; I’m puzzled as to why they wouldn’t offer several lower-spec models as the entry-level into and then progressive upgrades within that line, since only some people need that 256GB feature differentiator of running frontier-level models on their MacBook Pro.
And yet, Apple had to drop base configs from their lineup. They weren't selling $599 Minis at cost. They could take someone over and inflict damage on competition.
And what happens if the market settles back down or the leading memory tech pivots away from what you invested all this capital and time chasing?
You'd need a very strong, very particular forecast to make such a costly bet. And conversely, it may say something about their internal forecasts that they're not making the bet.
> And conversely, it may say something about their internal forecasts that they're not making the bet.
It says they are no longer worried about being punished for monopolistic behavior and have bet a “ballroom donation” will exempt them from another round of punishment.
I feel like folks around here have already forgotten about the last time the memory suppliers quietly agreed to keep raising prices and stop competing with each other.
> And conversely, it may say something about their internal forecasts that they're not making the bet.
Idk if you can read into it that way.
All these companies have cafeterias but you don't see them investing into farmland so they can get their bananas a few cents cheaper.
But also why bother spending 20B on a fab when you can invest 20B into TSMC and let them build the fab?
Probably for strategic reasons.
Mainland China is one concern.
Another is AI being the commodity compliment to semiconductors.
Because having every major company in America’s eggs in one basket is as insane as it gets, especially with China bearing down on Taiwan.
That’s an entirely different framework, one that doesn’t concern investment decisions of Apple or Google.
I do agree that USA and EU could foot the bill and subsidize a couple billions in such industrial infrastructure, perhaps taking back a cut of the profit rather than privatizing all of it.
But they’re not doing it, or are making pitiful efforts at that
It's just an insane amount of money to invest with the long-term effect of you oversaturating the chip market.
Throwing 20B into a chip fab in the EU would be politically a very unpopular move, if it's done as a public company or worse directly state owned, you'll royally piss of Taiwan, South Korea and China and it's likely they'd retaliate by e.g. subsidising their auto industry more in order to give the death blow to the EU auto industry.
It shouldn't be a politically unpopular move. Other countries can get pissed off, but those other countries also cannot guarantee that Europe will get chips when times are tough. If you want to build modern drones and missiles you need access to a large amount of computer chips. In a crisis will those still be available to Europe?
They'd more likely split themselves laughing.
The solution to threats to global economic integration is to address the threats to global economic integration. It's not to cannibalise our own full-employment high value economies, by diverting enormous capital and labour into duplicating vast swathes of lower value jobs we don't actually have the work force for anyway, just so we can pay unaffordable prices for the resulting goods.
We probably both agree it's an absurd fantasy, and the people trying to make stuff like this happen are implementing policies that ensure that it won't, such as putting tariffs on the inputs they need to build out this domestic manufacturing capacity in the first place.
It’s one thing when toys, pots, furniture etc. is made somewhere else. It’s a completely different thing when your high tech is manufactured there.
> The solution to threats to global economic integration is to address the threats to global economic integration.
So permanent world peace. That sounds much easier.
Amd it isn't only geopolitical threats we have to worry about. The world's hard disk supply disappeared with a tsunami in Thailand. Taiwan is vulnerable to those and earthquakes. Efficiency and robustness are at odds and we are leaning too far towards efficiency. Even if China hadn't been so large it could absorb the costs of capturing the world's entire manufacturing base with subsidies, centralizing that much has risks completely apart from politics.
The issue with national investment in the EU is that it might be attacked as state sponsored activity where one can complain that public money are used for market moves or if it is EU wide initiative, then the governments will squabble whose economy will get the money. The framework is not mature enough to allow for delegating to someone to solve the problem for everyone and to balance the beneficiaries in the long term.
Meta spent more on the Metaverse, it’s all the most certainly something they can afford to take a hit on and they won’t because memory and CPU usage is only going to go up from this point on.
Why should society care about people making profits? Society would greatly benefit from cheap abundant ram than FAANG shares going up. I'm kinda sick about only caring that some billionaire makes more money and would rather you know... actually improve conditions to better society.
This is why China is eating the West. Quite easy to start an electronics company when you have such an abundance of suppliers, compare this to America where there is maybe one or two players in the entire nation.
Quite pathetic, but we live in a pathetic world so it tracks.
You think Chinese businesses aren’t in it for the profits?
While these private companies exist and are in it for the profits, they don't control the government, meaning any potential price gouging is strictly regulated. The Chinese government actively prevents private companies from becoming too powerful or dominating an industry in the public eye. Instead, they foster competition by promoting and building up alternative companies.
A great example of this is how they handled digital payments. While Visa and Mastercard maintain a monopoly in the West, China faced a similar situation when private mobile payment systems began dominating the market. In response, the government stepped in and forced the ecosystem to open up to competitors.
I wouldn't characterize it as forcing to "open up". It's true they put an end to some of the most egregious anti-competitive practices, but the Chinese tech ecosyatem, including payments, is still notoriously non-interoperable. For example, there's no way to transfer money between WeChat Pay and Alipay.
The relationship between Tencent/Bytedance/Alibaba is what happens when there are no monopoly laws or they are ineffective: every company fighting tooth and nail to corner every market with a single "super-app". No specialization at all and many races to the bottom, like the recent war between Alibaba, JD and Meituan over instant delivery that made every company involved lose a lot of money.
But not open up to competitors like Visa.
Credit card processors are, as a natural aspect of their purpose, a deeply useful surveillance network.
Correct. A level playing field for all companies regardless of other factors is explicitly a non-goal.
In China the desire for profits serves the society. In the US the society serves the desire for profits.
They are but the difference is that China doesn't want to encourage monopolies and have zero qualms in jailing or executing bad business leaders.
There absolutely are monopolies. There are, in fact, many state run enterprises. Where do you get these ideas?
My understanding is that the Chinese government prefers to have multiple successful companies in each field and they don't like it when one company becomes too powerful.
They pivoted away from that quite a while ago now they just disappear people for a week or two and randomly completely crush a business here and there to make sure they understand their place.