Apple Silicon and Virtual Machines: Beating the 2 VM Limit (2023)
khronokernel.com160 points by krackers 7 hours ago
160 points by krackers 7 hours ago
This is a very silly restriction, at least to apply uniformly to all Macs. I think if you buy a more powerful Mac they should let you virtualize more Mac instances. Like an M5 maybe limit to 2, but maybe let an M5 Pro do 4 and an M5 Max do 8 or something.
Why should they impose a limit at all? Your hardware is a natural limit, you'll stop of your own accord when you reach its thresholds.
>Why should they impose a limit at all? Your hardware is a natural limit
because imposing an artificial limit keeps them from exposing how low the natural limits turn out to be? Apple Silicon need always to be spoken with reverence, ye brother of the faith, do not fuel the faithless lest they rend and threadrip that which we've made of wholecloth.
> Your hardware
Ah but when you buy an iPhone or a Mac, Apple sees it as their hardware graciously made available to you for a limited time and under ToS.
> Why should they impose a limit at all?
Whenever I see apple silliness, I have to remember:
"You're not the target market."Rent seeking, of course. They want to charge you for every physical and logical machine you use. Virtualization gets around that.
They'd probably charge separately for every feature of the processor if they could.
That would make more sense except they don't even have an option to pay for it.
Yes they do. It's called "another Mac". And I'm not even being snarky here: I legitimately think someone at Apple thought this through and said "yeah if they need more than 2 VMs running at the same time, there are probably multiple users and they can each get their own Mac".
Nah, Apple has been extremely restrictive about virtual machines in all kinds of ways, e.g. the minimum terms anyone is able to lease out a VM or Mac to someone else is 24h, making cloud-like workloads practically impossible. For some reason, Apple really doesn’t like virtual machines, and it’s much more intentional than just “probably multiple users”.
It’s extremely frustrating.
The option is you have to buy another machine. There are mac ec2 instances and several mac cloud hosts that all would abuse this if they could, instead to stay compliant they buy more machines.
And thus they need a massive datacenter full of systems, rather than a pile of paid licenses.
And macOS remains a toy for use only by individuals that is a massive pain for developers to support.
They are likely scared of people who would run MacOS virtual desktop farms, without also buying an appropriate number of Apple machines.
That’s what I would be worried about if my primary source of income was hardware sales.
IMO they should sell appropriately priced licenses that allow the use of more VMs. Make the licenses expensive enough so that it doesn't eat into hardware sales, or explicitly prohibit VDI/virtual seats in the license agreement.
Currently services like Github Actions painfully and inefficiently rack thousands of Mac Minis and run 2 VMs on each to stay within the limits. They probably wouldn't mind paying a fee to run more VMs on Mac Studios instead.
Apple had to be dragged kicking and screaming into the world of virtualization and the idea of macOS running on anything besides "metal built by Apple." They've been pretty clear for decades that they only care about customers who buy Apple aluminum and silicon.
Imagine buying a mac studio with 500+ GB of memory and being limited to 2 vms.
Yeah that is what I was going to do until I discovered the two VM limit. I was building a MacOS GitHub Actions farm, or rather, looking into it. I had written most of the code but my inertia screeched to a halt when I discovered the two VM limit for MacOS VMs.
They discontinued the 512GB Studio, and the Pro is gone, so no fear there now.
They still EXIST though. And I saw one the other day on the Refurbished store. They’re definitely still around.
Even a 256GB model would run a load of 16GB VMs
Market design.
They don't want to be in the server business, they don't want there to be third party VM providers running Mac farms selling oversubscribed giving underpowered disappointing VM experiences to users who will complain.
A bunch of folks want Apple to enter a market Apple doesn't want to enter into. They have tools available which would enable that market which they are kneecapping on purpose so that nobody unwillingly enters them into it. The "two VMs per unit hardware" has been in their license for at least a decade.
The limit isn't really a resource issue, since you can run pretty much an "unlimited" number of non-Mac VMs. I suspect it's more of a business decision, such as preventing people from setting up shop as a low-cost Mac VPS provider.
Maybe it doesn't work. Why are you so sure it would? It may perform very badly.
I buy a $100 Windows 11 Pro licence, and my limit is 1024 VMs
Hyper‑V on Windows 11 supports up to 1024 simultaneous VMs per host if the hardware can handle it. On my little Windows ARM laptop I can easily run 4 VMs before it runs out of steam.
On Mac, you can run lots of Windows/Linux VMs and two Mac VMs.
On Windows, you can run lots of Windows/Linux VMs and zero Mac VMs.
The limit of 2 is just for virtualizing macOS. You can run as many Linux VMs as you want at once on macOS.
There's first class support for Linux on Windows, and Microsoft has a developers VM available for download so you can run as many Windows as you want. I do a Hyper-V Quick Create and there are three flavors of Linux to choose from, or Windows, with all the development tools pre-installed.
The only reason Linux exists on Windows is they're trying to redo the 90s playbook of dominating then destroying the competition. I was almost on board in the Windows 10 era, switching a whole lot of my time to doing things in WSL on Windows.
Windows 11 and the walled garden greed they're trying to enable is so bad that this dominating Linux attempt is certainly failing, the only reason I haven't completely ditched my Windows system is that my several TB external drive is at large and I haven't taken the time to actually do it.
Plus Steam and their Wine work is absolutely killing it so the one thing that was keeping me motivated to still have a Windows presence is pretty much gone.
It really is silly. The other day I decided to try this openclaw thing out but concerned about the security stuff, so I took VM for a spin only to find out the iCloud and the App Store were restricted.
Seems Mykola Grymalyuk started working at Apple 2 years after this blog post. You either die a hero..
starting with M3+ you can use Hypervisor.framework/Virtualization.framework to spin up nested VMs.
it would be amusing if that bypassed the limit.
Lol with 2 VMs per VM you can do an infinite VM linked list where each macOS hosts a "guest" and a "next host". I'm too lazy to test this out. Any takers?
I think it's a little funny that my response is "no I'm not wasting my weekly tokens on that, it's not a good enough bit"
I’m very curious, why did Apple put such a limitation?
Because their business model is to sell tightly integrated hardware and software as a package. The hardware sales fund the software development. They don't want people who haven't bought the hardware using the software.
The VM limit only applies to the number of macOS VMs launched from macOS itself.
My 2018 mac mini officially supports VMware ESXi to be installed directly on the hardware and virtualize any number of macOS machines
Funny enough I can even launch more than 2 macOS vms on my framework chromebook with qemu + KVM from the integrated Linux terminal.
macOS is proprietary software. You need a license for every copy you run, whether it's in a VM or not. The VM limit is written into the macOS EULA.
> to install, use and run up to two (2) additional copies or instances of the Apple Software, or any prior macOS or OS X operating system software or subsequent release of the Apple Software, within virtual operating system environments on each Apple-branded computer you own or control that is already running the Apple Software, for purposes of: (a) software development; (b) testing during software development; (c) using macOS Server; or (d) personal, non-commercial use.
This implies anyone doing this using VMware violates the EULA?
Yes. Apple's not going to come after you for running too many VMs on your personal machine, but if you're running a commercial enterprise involving macOS VMs they do care.
VMware vSphere is not a product intended to be used by consumers. It's intended to be run by enterprises at scale. ESXi is running the vms not macOS.
https://i0.wp.com/williamlam.com/wp-content/uploads/2020/04/...
Yeah but the "hardware" in that sense is almost entirely iPhone and iPhone-adjacent, Mac is a trailing 4th- or 5th-place line of business... maybe 6th.
Probably to prevent a single hardware system from being used to run an online identity farm.
Doesn't make too much sense, the VMs don't get unique hardware identifiers that one could (ab)use for spamming iMessage.
That kind of tracks as the source of the concern. My first thought was it’d be something IDMS-related as well. I don’t know enough about that system to pinpoint exactly what.
MacOS is full of these anti-owner decisions. They want full control over your experience for their benefit.
This is a really cool article, but the existence of such an arbitrary limit on any serious development platform is weird.
Has apple been a serious development platform in the last 20 years?
I know a lot of devs like apple hardware because it is premium but OSX has always been "almost linux" controlled by a company that cares more about itunes then it does the people using their hardware to develop.
At least 9 out of every 10 software engineers I know does all their development on a mac. Because this sample is from my experience, it’s skewed to startups and tech companies. For sure, lots of devs outside those areas, but tech companies are a big chunk of the world’s developers.
So yea I would say Apple is a “serious development platform” just given how much it dominates software development in the tech sector in the US.
I have the feeling a lot of people take Macs because the other option is a locked down Windows, and Linux is not offered.
This. I ran Linux at work until last year, when it was finally disallowed. I went with locked-down Mac over locked-down Windows.
The hardware for a Linux laptop right now is not great. Especially for an arm64 machine. Even if the hardware is good the chassis and everything else is typically plastic and shitty.
That is a surprising sentiment. Most dell and Lenovo laptops work just fine and are usually of reasonably good build quality (non-plastic chassis etc.).
arm64 is however mostly bad. The only real contender for Linux laptops (outside of asahi) was Snapdragon's chips but the HW support there was lacking iirc.
They give us Dell Linux machines from work. They suck so bad and we have so many problems. Overheating, camera is terrible, performance is bad relatively to the huge weight of the device. Everything is a huge step down from Macs.
Whenever I see Linux people comparing Linux and Mac I'm amazed at the audacity. They are not in the same league. Not by a mile. Even the CLI is more convenient on the Mac which is truly amazing to me.
What happened to all the love for Framework?
The honeymoon of Lego-brick replaceable USB ports is over?
I have a personal Framework 13 and a work-issued MacBook Pro. I love Framework’s mission of providing user-serviceable hardware; we need upgradable, serviceable hardware. However, the battery life on my MacBook Pro is dramatically better than on my Framework. Moreover, Apple Silicon offers excellent performance on top of its energy efficiency. While I use Windows 11 on my Framework, I prefer macOS.
Additionally, today’s sky-high RAM and SSD prices have caused an unexpected situation: Apple’s inflated prices for RAM and SSD upgrades don’t look that bad in comparison to paying market prices for DIMMs and NVMe SSDs. Yes, the Framework has the advantage of being upgradable, meaning that if RAM and SSD prices decrease, then upgrades will be cheaper in the future, whereas with a Mac you can’t (easily) upgrade the RAM and storage once purchased. However, for someone who needs a computer right now and is willing to purchase another one in a few years, then a new Mac looks appealing, especially when considering the benefits of Apple Silicon.
>>At least 9 out of every 10 software engineers I know does all their development on a mac
I work in video games, you know, industry larger than films - 10 out of 10 devs I know are on Windows. I have a work issued Mac just to do some iOS dev and I honestly don't understand how anyone can use it day to day as their main dev machine, it's just so restrictive in what the OS allows you to do.
It makes sense that you use Windows in a video game company. We use windows as well at work and it's absolutely awful for development. I would really prefer a Linux desktop, especially since we exclusively deploy to Linux.
Weird .. macOS is still completely open is my experience. Can you give an example?
I compile a tool we use, send it to another developer, they can't open it without going through system settings because the OS thinks it's unsafe. There is no blanket easy way to disable this behaviour.
We also inject custom dlibs into clang during compilation and starting with Tahoe that started to fail - we discovered that it's because of SIP(system integrity protection). We reached out to apple, got the answer that "we will not discuss any functionality related to operation of SIP". Great. So now we either have to disable SIP on every development machine(which IT is very unhappy about) or re-sign the clang executable with our own dev key so that the OS leaves us alone.
If SIP is kicking in, it sounds like you're using the clang that comes with Apple's developer tools. Does this same issue occur with clang sourced from homebrew, or from LLVM's own binary releases?
If it's being sent to another developer then asking them to run xattr -rd com.apple.quarantine on the file so they can run it doesn't seem insurmountable. I agree that it's a non-starter to ask marketing or sales to do that, but developers can manage. Having to sign and then upload the binary to Apple to notarize is also annoying but you put it in a script and go about your day.
But Apple being "completely open", it is not.
I work as a consultant for the position, navigation, and timing industry and 10 of 10 devs were on Windows. Before that I worked for a big hollywood company and while scriptwriters and VP executive assistants had Macs, everyone technical was on Windows. Movies were all edited and color graded on Windows.
Webshitters don't "engineer" anything, it's insulting you would insinuate that.
Anyone who watched the Artemis landing yesterday would have been keen to notice all the Windows PCs in use at Mission Control — nearly all hosting remote Linux applications.
Not a Mac in sight.
They were using VLC on Windows in space.
If all the Macs in the world disappeared tomorrow, everything essential would somehow continue unabated.
> Has apple been a serious development platform in the last 20 years?
i dont think anyone asks this question in good faith, so it may not even be worth answering. see:
> I know a lot of devs like apple hardware because it is premium but OSX has always been "almost linux" controlled by a company that cares more about itunes then it does the people using their hardware to develop.
yea fwiw macs own for multi-target deployments. i spin up a gazillion containers in whatever i need. need a desktop? arm native linux or windows installations in utm/parallels/whatever run damn near native speed, and if im so inclined i can fully emulate x86/64 envs. dont run into needing to do that often, but the fact that i can without needing to bust out a different device owns. speed penalty barely even matter to me, because ive got untold resources to play around with in this backpack device that literally gets all day battery. spare cores, spare unified mem, worlds my oyster. i was just in win xp 32bit sp2 few weeks ago using 86box compiling something in a very legacy dependent visual studio .net 7 environment that needed the exact msvc-flavored float precision that was shipping 22 years ago, and i needed a fully emulated cpu running at frequencies that was going to make the compiler make the same decisions it did 22 years ago. never had to leave my mac, didnt have to buy some 22 year old thinkpad on ebay, this thing gave me a time machine into another era so i could get something compiled to spec. these techs arent heard of, but its just one of many scenarios where i dont have to leave my mac to get something done. to say its a swiss army knife is an understatement. its a swiss army knife that ships with underlying hardware specs to let you fan out into anything.
for development i have never been blocked on macos in the apple silicon era. i have been blocked on windows/linux developing for other targets. fwiw i use everything, im loyal to whoever puts forth the best thing i can throw my money at. for my professional life, that is unequivocally apple atm. when the day comes some other darkhorse brings forth better hardware ill abandon this env without a second thought. i have no tribalistic loyalties in this space, i just gravitate towards whoever presents me with the best economic win that has the things im after. we havent been talking about itunes for like a decade.
It is a weird situation. Apple products are consumer products but they make us use them as development hardware because there is no other way to make software for those products.
Making software for other Apple products pretty low on the reasons I use a MBP.
128GB of RAM and an M4 Max makes for a very solid development machine, and the build quality is a nice bonus.
Anything being developed for the Apple ecosystem requires use of the Apple development platform. Maybe the scope could be called "unserious," but the scale cannot be ignored.
I am aware.
However having used Xcode at some point 10 years ago my belief is that the app ecosystem exists in spite of that and that people would never choose this given the choice.
For me at least, not being Linux is a feature. Linux has always been “almost Unix” to the point where now it has become its own thing for better or worse. OS X was never trying to be Linux. It would be better if we still had a few more commercial POSIX implementations.
That is fair but in my experience most devs are targeting linux servers not BSD(or any other flavour) which is helped by OSX. If OSX was linux derived it would suit them just as well.
edit: I suppose I should also note the vast majority of people developing on mac books (in my experience anyway) are actually targeting chrome.
There is no reality that macOS could be based on Linux.
Turns out, an operating system is more than just a kernel with some userspace crap tacked on top, unlike what Linux distros tend to be.
> Turns out, an operating system is more than just a kernel with some userspace crap tacked on top, unlike what Linux distros tend to be.
This is also my opinion of OSX, let's not pretend that the userland mess is the most beautiful part of OSX.
Apple has great kernel and driver engineering for sure but once you go the stack above, it's ducktape upon ducktape and you better not upgrade your OS too quickly before they fix the next pile they've just added.
Heterogeneity is the feature. The Linux ecosystem is better off for it (systemd, Wayland, dconf, epoll, inotify are all based on ideas that were in OS X first) and not being beholden to Linux is a competitive advantage for Apple everyone wins.
> I suppose I should also note the vast majority of people developing on mac books (in my experience anyway) are actually targeting chrome.
Point taken. Most developers probably make do with Linux containers rather than MacOS VMs.
Apple had real Unix a decade before the Linux crap was made, a bad unix copy. Nextstep was much better than Linux crap. "A budget of bad ideas" is what Alan Kay said about Linux [1], he invented the personal computer.
My 1987-1997 ISP was based on several different Unix running on Apple, probably long before you where born.
Apple built several supercomputers.
[1] https://www.youtube.com/watch?v=rmsIZUuBoQs
[2] Founder School Session: The Future Doesn't Have to Be Incremental https://www.youtube.com/watch?v=gTAghAJcO1o
Are you talking about A/UX? That was one of the first Unix systems I was exposed to.
Alan Kay invented a dead end (smalltalk). Meanwhile Linux became the future.
Apple had a terrible Unix until they bought NextStep.
Yeah, they were that, and for the last 20 years they have been the iphone company.
> When using a custom kernel collection with Apple Silicon, there are some unfortunate downsides. The biggest being that streamlined OS updates are no longer available.
This might be a blessing in disguise.
It’s crazy that you can compile a custom kernel and it’ll boot and the GUI will run.
Can this work with lume as well? Currently it has a similar limitation.
it should, lume is a thin wrapper around Apple's Virtualization.framework as i understand it
Very funny to see HN hate on Microsoft and Google but then love a company where they cannot even run an app on their mobile platform without Apple's permission or only a certain number of VMs on the hardware they own .
Someday I may be able to retire this link, but today is not that day: https://en.wiktionary.org/wiki/Goomba_fallacy
HN is not one person. I'm very happy to hate on all of them. I see what you mean though. I've given up on getting normal people to care, but seeing programmers who are absolutely smart enough to run their own Linux system on computers they actually own actively choose not to do so is very disconcerting.
>seeing programmers who are absolutely smart enough to run their own Linux system on computers they actually own actively choose not to do so is very disconcerting.
I run macOS because Apple understands that QA testing is something of actual importance, and designing yet another package manager is not.
I do spin up Linux every now and again to see if it's good yet, and always walk away.
Why do documents print at ~50dpi on my network printer?
Why does the system simply not wake up ~20% of the time when I open my laptop's lid?
Why do I have to unplug and reconnect my USB WiFi Dongle every hour or so when the internet randomly drops out?
Why does the system stop recognising my USB SD Card reader occasionally, forcing me to hard reboot the system?
Why is the audio distorted over HDMI when I enable HDR?
Why does Kodi only detect a refresh rate of 30Hz when the system itself has no issues seeing that the monitor is 60Hz?
All of these are real problems that real users have had, but instead of solving them the Linux development community instead chooses to devote their time and resources navel gazing about systemd alternatives or creating a fragile AUR package for software that already has a sensible and officially supported distribution method.
All operating systems have bugs, and Apple doesn't have the QA it used to have. MacOS has basically been exclusively trending down in quality for a while now, while Linux continues to get better.
What you have to realize is that what Linux distros are doing is inherently more complicated. They're making a general purpose operating system intended to run on every computer.
Apple is making one operating system intended to run on maybe 0.1% of devices. Oh, and they also make those devices.
And MacOS is still trending down in quality, somehow.
You're not wrong about the downwards trend in quality but we're still a long ways off from macOS or even Windows having the same level of QA issues that Linux does, on a regular desktop system.
TL;DR you sacrificed your freedom for convenience, you think quality assurance is worth being at Apple's mercy, you signed away the keys to "your" machine so they can "manage" it for you along with the rest of your life.
Meanwhile I'm running about a dozen of development virtual machines right now. I'm limited only by the amount of RAM my computer has. It never even occurred to me that some gigacorporation out there would have thought to limit the VMs their own users can spawn. Every day, they reach a new low.
On the other hand I’m very conveniently enjoying my experience, I don’t have to waste time screwing with stuff I have no interest in screwing with - like the OP’s examples, and if I want to run Linux I’ll just install it and do what I want or rent out some compute time somewhere.
Besides, you can buy a Mac and do whatever you want and go buy a bunch of off the shelf components to do whatever hobby stuff you want to do too.
Freedom, perhaps, starts with not making up and applying limitations on yourself.
> Freedom, perhaps, starts with not making up and applying limitations on yourself.
Nothing wrong with applying limitations to oneself. That's discipline, principles. It's important stuff.
The real problem is accepting the completely made up limitations that others apply on you. Corporation wakes up one day and just decides people can't run more than two virtual machines? That's stupid. Actually defending this with "but convenience" arguments as if convenience was supposed to override freedom? No.
Freedom isn't something you actively work towards. It's something you start with. It's the status quo. Others take it away from you. You can either accept it passively and enjoy the "convenience", or you can resist and go down the harder path. It's very disappointing to see people on Hacker News choose the former path.
You’re just living under the illusion of freedom. You are completely dependent on the decisions of others and their good graces for all of your computing needs, from the silicon to the Linux distro you use. You’re just drawing an arbitrary line a little further to feel like you’re in control, but you’re not.
Silicon? Sure. Billion dollar fabs are huge single points of failure. It's turning into a problem too due to the war on general purpose computing. Free software doesn't matter if we can't run it. Linux distro? Not really. It's only a matter of how much effort I want to put into things. I can make my own distro, I can't make my own trillion dollar fab.
Anyway, what even is this argument? Can't control everything, so it doesn't matter? Don't even bother trying? Just give up? Just accept your lot in life as a serf in Apple's digital fiefdom? I'm pessimistic about the future but even I haven't completely succumbed to such total nihilism yet.
The VM limitation is only for macOS guests, otherwise I can spin up as many VMs as I like, which is no different to doing so in Linux (since it cannot run macOS VMs).
>TL;DR you sacrificed your freedom for convenience
Yes I did, just like you did when you chose to live as a taxpaying member of society rather than a hermit scouring the bush for berries and fish.
Enjoy your VMs.
Living as a taxpaying member of society is something that is imposed on us. If we refuse, violent men with guns show up at our doors to arrest us and seize our property. At least we get to try and vote out idiots imposing stupid quotas on the population.
The issue of computer freedom does not even come close to this. None of this is imposed on us. We have the power to choose differently at any time. We can choose not to accept the monopolistic corporation's terms.
I use a Macbook for work and do all my development via ssh on remote Linux instances. Each OS is doing what it does best. I last tried a Linux laptop for development in 2020 and my conclusion was the same as in 2010: never again for at least a decade. I have better things to do than fix broken drivers and curse at shitty trackpads.
Since when are users in this place shy about bashing Apple?
Plenty of hate out there of apple alongside the love.
In the very same comments sometimes, those frustrating geniuses
Inside of me are two wolves. One that’s like “F Apple” and another that is like “Are they going to do an M5 ultra or…?”
We can appreciate their hardware achievements and at the same time condemn them for their monopolistic anti-user decisions.
Adults can hold 2 thoughts in their head at their same time.
Yes, indeed, complaining about them even though they're brilliant (tough love?) as I just did
Paraphrasing F. Scott Fitzgerald? "The test of a first-rate intelligence is the ability to hold two opposed ideas in the mind at the same time, and still retain the ability to function."
Holding contradictory ideas isn't the laudable skill. Any uncritical person can believe conflicting things without being troubled by them. The genius is holding such ideas in disbelief long enough to let evidence alter or evict them.
Please avoid these kinds of sneers that characterize the whole community as being united in “hate” or “love” for any particular company or technology.
HN is a diverse global community and its views about most topics form a normal distribution, and most people here are able to form nuanced opinions that consider the positives and negatives in all these topics. This kind of “very funny” swipe relies on a caricature that's easy to portray if you focus on the loudest voices on one side of any discussion but falls away if you make the effort to read the discussions in depth.
IIRC you can just turn off sip and set the boot argument that controls it without a custom kernel
