WSL 2 is getting faster Windows file system access
boxofcables.dev179 points by haydenbarnes a day ago
179 points by haydenbarnes a day ago
Proton, Copilot, and literally this single issue are what pushed people to Linux. If I were in charge there would be a team devoted to fixing this a decade ago.
WSL singlehandedly stemmed much of tide of developers moving away from Windows, but WSL native filesystem performance gave devs that magical experience when they boot into Linux the first time and see that the filesystem doesn't have to be ass. There's always been hacks around this, but for many devs the easiest hack was to ditch Windows.
They should have moved heaven to fix this on day one, there's really no engineering excuse. Linux is open source.
Onedrive constantly trying to steal all my files, bing in the start menu, windows update hogging resources then rebooting at the worst time, offline updates taking fking forever even with a fast SSD, layers and layers of bloat and garbage we have to click through or remove on new installs, removing customisation features and taking a decade to half-ass a control panel rewrite, I could go on...
> Onedrive constantly trying to steal all my files
By default setting, windows defender will upload "suspicious files" to Microsoft.
A while back I caught it trying to nab my OpenVPN installer which also contains the certificates.
I turned sample submission off permanently via group policy after catching defender uploading places.sqlite out of my Firefox profile.
Basically the Apple and Google stuff that others like so much.
All 3 (Apple, Google, Microsoft) share a lot of the same negative behaviors, but only Apple and Google get a free pass for some reason. Microsoft is worse in many aspects, but look at the recent debacle with NightmareEclipse and how shitty MSRC is. Apple pulls the same crap and are even less transparent about security, but they get a free pass in tech circles for some reason.
I don't think Google gets a free pass--kind of the opposite, it gets the lion's share of the criticism (especially for privacy stuff) despite being the most open and up front about what they're doing, since they open source most of their operating system.
I don't think there's any company that's done more to support transparent open-source software except Red Hat. Hell, in true Google style, they've built not one, not two, but four separate FOSS operating systems (Android, ChromiumOS, Fuchsia, and at this point it's time to admit Chromium itself has become an OS-within-an-OS), as well as being the second-biggest contributor to Linux.
I don't think Google gets a free pass--kind of the opposite, it gets the lion's share of the criticism (especially for privacy stuff) despite being the most open and up front about what they're doing. (If you want to know what data Chromeium is collecting, just check the source!)
I don't think there's any company that's done more to support transparent open-source software except Red Hat. Hell, in true Google style, they've built not one, not two, but four separate FOSS operating systems (Android, ChromiumOS, Fuchsia, and at this point it's time to admit Chrome has become an OS-within-an-OS), as well as being the second-biggest contributor to Linux.
> Proton, Copilot, and literally this single issue are what pushed people to Linux.
This isn't the only issue. I think another big issue is pushing more and more integration with Microsoft cloud services (e.g. Microsoft accounts), advertising, etc, which Microsoft has made increasingly difficult to opt-out of. They could fix every single technical limitation anyone has ever complained about, but if they don't change their corporate culture on forced cloud/advertising/etc, many won't care about those fixes.
That their solution is a few winget scripts on github to turn a vanilla Windows install into a "developer optimized" install is telling.
They could have, instead, made a standalone ISO of Windows with all the crap already stripped out. Instead, they are still treating the crap-filled version as the default with an option to strip it down.
If they wanted to correct their reputation, it needs to be the other way around. Ship the stripped down version as standard, and make the crap opt-in.
Yeah, my switch to Linux and Mac for most things is more about just finding Microsoft's policies so obnoxious and hostile that I just won't deal with them anymore, even if I have to deal with more technological hassles. The only reason I haven't completely nuked my Windows partition is because I can at least use Rufus to turn off the worst stuff. But frankly, the amount of software that keeps me on Windows is dwindling fast, and every time Windows update resets my browser to fucking Edge or signs me into a Microsoft account system wide without my consent I just get that much closer. It feels like malware at this point.
Just a comment on Proton... I've recently shifted to linux (garuda ) as a native OS for gaming (still dual booting, but linux is my main OS now, I used to run linux VMs in windows). My experience with Proton is that only ~30% of my games work out of the box. Some games like dota2, and factorio are native linux and work MUCH better (faster/higher fps) in linux. A bunch of windows games work fine, other's semi work, and I have to spend a bunch of time investigating why I'm getting the issues I'm getting. Others just aren't really supported (it seems anti cheat software is a big blocker) or I just can figure out what is going wrong quick enough that I just abandon it. Overall, everything seems better in linux world, everything is really snappy. I'm hoping more game companies treat linux as a first class citizen as more people switch. It is definitely a great platform for gaming but really just needs game creators to ensure their games work, ideally native, but even just using Proton would be good.
Outside of games using anti cheat, for me the most common fix is to just select Proton Hotfix or Proton Experimental in the game's compatibility settings.
For some reason certain games default to specific older Proton versions. And then of course there are a bunch of older games with a native Linux port that is now unmaintained and stopped working at some point, but they still work just fine with Proton :)
Anticheat is indeed a huge blocker, and given how invasive shady kernel anticheat software is on Windows I kinda hope it stays that way
I think even Microsoft is getting ready to be done with kernel level anticheat. After crowdstrike they've been planning to kick everyone out of Ring0, making new user mode APIs. They're focused on EDRs right now but have publicly said eventually everyone is going to be kicked out of the kernel.
Isn't Garuda an arch distribution? It could be that (less common, so more issues). Running SteamOS likely has better chance of running games, but yeah the linux experience is not streamlined at times (but it is functional!)
SteamOS is also an Arch distribution, though I'm not sure how significant the changes are. AFAIK Lutris uses Ubuntu's libraries for better compatibility across distributions, maybe Steam does something similar.
kinda, SteamOS is an arch distribution but it's locked down by default (you can access stuff) and it's run by the company selling games on PC, so they make it work. On a normal arch distribution, you have to deal with installing dependencies yourself, so more of a pain.
It's very very rare for me that games don't work. It's almost all competitive games, where the game specifically does not allow anti-cheat.
There's very little fiddling around or configuring. 30% sounds god awful terrible; my success rate definitely >85%. In the rare case something doesn't work right away, https://www.protondb.com/ usually has advice in the top or second comment that works great.
I don't really think the windows vs Linux native debate is worth pursuing. Windows games run better than they do on Windows 4 times out of 5, and that's more than good enough.
What games don't work? I've never had a game I wanted to play but couldn't due to being on Linux.
Perhaps you meant Pluton and not Proton? https://news.ycombinator.com/item?id=25191319
Proton is a tool Valve made, based on Wine, to easily run Windows games, on Linux [0]. GP meant Proton.
Sorry, I thought you were enumerating gerivances against Microsoft and not better alternatives. Pluton has been controversial because it's been marketed as a "security" solution when in reality is's just doing DRM. Originally greated for the Xbox, it's kind of the antithesis of Valve's Proton.
Tangentially , I was a heavy used of wsl and moved to linux a few months ago and LLMs made most of the downsides of using linux as a desktop go away for me. I chatted with claude about the migration to find the best distro, decided on Fedora. After the install I asked everything I wanted to configured and got straight answers. In 3 or 4 hours I had an even more comfortable experience than I had on windows. AI made the annoying parts of trying to figure out how to edit all the config files to have linux behave the way you want very easy. I also had claude code write a bunch of scripts that I could have done but would probably never bring myself to actually do it . WHen you have a coding agent readily available , having an open source desktop environment makes a lot more sense. I encourage everyone to try it.
I also did this as well as learned pfSense then OPNSense when pfSense went bad. Also made a pretty complicated XCP-ng setup. Learned all this with the ancient ChatGPT 3.5-4.0 models.
I can hear a subset of people cringe saying "but LLMs are BS machines and you aren't learning anything!" I heartily disagree on both fronts. The main thing holding users like me back from linux was always the snarky RTFM community and the fact that everything has 25 different answers (depending on distro, window manager, and many other factors). LLMs take care of all this friction for you very nicely.
Linux has a lot of inherent advantages here, like much deeper terminal support that let an agent make changes to practically any system setting on your behalf. Most importantly, if all else fails, the LLM agent can always check the source code to debug what's going on at the root, if you're using Linux. They don't have the option in Windows or MacOS.
This could mean that at some point, ironically Linux becomes the most user-friendly operating system for regular people as their AIs can diagnose and fix system issues and problems for them more easily than they can with proprietary operating systems.
On the other side, I am a lifelong Linux user, and even with advanced LLMs, trying to get Microsoft Windows to behave sanely takes hours every month for years on end (thanks, day job). Things Linux figured out in 2003 are still magic or completely undoable on Windows.
Not my experience. I installed the IoT version of Windows 11 and have changed exactly one setting (put the start menu back on the left where it rightfully belongs). Nothing else has required any changes at all to get sanity, which is more than you can say for Mac (which requires half a dozen extra apps to make it same - SteerMouse, Karabina, Spectacle, etc.), or Linux, where the UI is mostly sane - if you're using KDE anyway - but you're beset by bugs and jank.
And don't say "it works for me". We know some people get very lucky with Linux and stuff just works. In my experience the typical experience is that stuff works much better on Windows than Linux.
E.g. something Windows figured out in 2000 - what happens when your system is low on ram or overloaded? Simple! Press ctrl-alt-del, it will pause other apps and allow you to open the task manager and choose one to kill.
On Linux? There's no task manager in the ctrl-alt-del menu (on KDE anyway), and even if there was it isn't a specially privileged UI so it wouldn't respond. Running low on RAM? No problem we'll just kill a random process and if that doesn't work (it usually doesn't), completely freeze and then hard-reboot. Yeay.
The Task Manager that many times also stopped responding or was permanently stuck behind a frozen full screen game?
On Linux, KDE has the plasma System Monitor, and if you can't use it for some reason you can switch your entire session to a TTY with CTRL+ALT+F2 and kill any process you want.
> you can switch your entire session to a TTY with CTRL+ALT+F2 and kill any process you want.
How intuitive. That's often disabled, and I've definitely had machines that were so frozen even that didn't work. I guess sysrq keys might have worked but if you're seriously going to suggest that you haven't understood the problem.
> The Task Manager that many times also stopped responding or was permanently stuck behind a frozen full screen game?
Hasn't happened once for me for as long as I can remember.
In fairness Linux has been pretty solid for me too after upgrading to 128GB of RAM and 64GB of swap. But I never needed to do that on Windows.
> Hasn't happened once for me for as long as I can remember.
I've had it happen to me, but the last time it did was probably around ~2017 or so on spinning rust. Fast SSDs make it a non-issue now.
This. It is hard to exaggerate how easy Claude Code (or, I'm sure, any number of other harnesses of choice) makes it to migrate to a new operating system.
It is truly a Star Trek-level experience. Nobody who doesn't want to run Windows (and who isn't forced to run it) needs to run Windows anymore.
What killed WSL for me was the incredibly janky way I had to share USB peripherals. usb-ipd works 80% of the time, all the time.
80% is being generous. I gave up on USB for WSL altogether and just pass the whole controller through with Hyper-V. On laptops, though, that is not always possible.
If it is as good of an improvement as the first major update, it will be hard to tell from native.
Hopefully, they will just push it out to everyone asap. We make heavy use of symlinks into Windows drives.
They are undoubtly doing this because so many users operate out of /mnt/c with zero clue of that implication.
Would you mind elaborating (for those of us uninformed)
/mnt/c is a mounted C: drive in WSL2, that allows WSL2 guests to read/write files on the Windows host.
The mount is fine and speedy enough, but the underlying reads/writes turn into native NTFS reads/writes through Windows. NTFS file API is incredibly slow - high fixed overhead for initial file access.
So patterns like node_modules with many small individual files (or compiling code in general) are much much slower on Windows or WSL2 /mnt/c due to the fixed overhead adding up over a large number of files.
It's a ridiculous problem that has plagued Windows for years.
It's the file system filters that are an issue on Windows. It trades performance for extensibility.
NTFS itself is a fast file system.
"NTFS itself is a fast file system"
File access is significantly slower on Windows compared to Linux on the same hardware. You can run Linux inside a VM and it will easily beat the host OS in filesystem performance.
If the problem is not NTFS, then it is in how the Windows OS uses it
> If the problem is not NTFS, then it is in how the Windows OS uses it
Isn't that exactly what the comment you are replying to is saying?
This is what I keep hearing. However, I have never had the good fortune of using a Windows installation where NTFS was allowed to be fast.
eBPF has opened the door to such bloatware on Linux. Previously there was no easy, stable way for enterprise bloatware vendors to maintain complex file and subsystem filter and analytic modules. Now that market is exploding because of eBPF, and it's a big reason there's so much work around growing eBPF's capabilities.
Could you expand? I never understood why this was considered acceptable, how is windows filesystem so slow compared to linux?
I am familiar with the issue, doing any sort of ruby development is a nightmare on windows because each require loads a file becoming increasingly slow at boot time
WSL2 operates through a file-level translation layer, not unlike NFS in Linux, which can be a huge performance problem for programs that manipulate lots of tiny files, like git or npm.
Programs that manipulate lots of tiny files are a performance nightmare on Windows even leaving aside the issue of WSL2.
git and npm install are somewhat slower on vanilla Windows, true, but on WSL2 they're downright unusable
One example is that if you have a node modules folder on Windows and you try to delete it from WSL it can take 10 plus minutes whereas if you deleted it directly in Windows it would have just taken a few seconds
Also if you try running Next js from files on Windows from WSL it takes minutes for each page to compile to the point that any local development is impossible so you would have to either run the Next JS server on Windows or move the files to WSL
Anything with node_modules takes ages on my Windows machine, whether it is through WSL, Docker or direct, largely in part due to corporate filters, checks, anti-virus and malware protectors and endpoint control.
Although the stuff you mention is true, it is not the only reason. NTFS is just notoriously bad at reading/writing tons of small files.
It also has a lot of problems with locking files that are open by a process, if you have a rogue process reading your node_modules npm install or rm node_modules can hang until that process finishes.
yarn2 keeps dependencies as tarballs instead of extracting them to disk, imported files get loaded from the tarball at runtime. Makes a massive difference in windows.
WSL2 is a VM based on a Windows virtual disk file (VHD). inside that VHD IO is quite fast , a couple degrees worse than native. /mnt/c is how you access your windows files, but it's slow like NFS (socket based). anything needing high IOPS will be dog slow e.g. compiles, file scanning, etc.
the rule of thumb without the newest features is to copy work to/from /mnt/c into $HOME as needed.
I was trying WSL years ago and this is one of the reasons I just moved to a full linux server instead. We still have way too many problems interfacing across filesystems. I hope with AI we will see an iteration on ExFAT that has all the journalling, versioning etc. magic of modern FS' and can be adopted across all 3 OSes. Probably a long shot but I can dream :)
Moved to mac about 7 years ago because of horrible WSL file system speed was.
You could just move your files to the WSL file system
That kind of defeats the entire purpose of them being accessible from the rest of the system.
You can access them from the rest of the system. For normal usage the performance is completely acceptable but for development tasks it matters.
People who run WSL are not normal users. The filesystem problems make Windows+WSL feel like a Trabant when you're used to a Porsche.
Yes well I felt the same way with how slow Docker file systems were on Mac but I think that's fixed now.
> Just copy it into the WSL file system
Yea bro totally. Totally. I'm gonna copy 2TB of media into the WSL virtual disk just so ffmpeg can run a little faster but still way slower than simply running linux.
(I beta tested the shit out of WSL1 and 2) before I wised up and just installed Gentoo forever.
You can run that directly on Windows.
But either way yeah most people aren't dealing with large media libraries that's obviously a little more difficult. But if you are primarily operating on them with WSL then you would just keep them in the WSL file system and you could access them from Windows whenever you need to...
Indeed. I have my agent edited files in podman in Lima, under two layers, and it's fine, because I do most stuff within my podman VMs. (I have shared volumes so I can review things before pushing the changes to my forge in separate containers that the agent can't access. When I need stuff on my mac, which is the exception, not the rule, I just copy them, putting them in a tar or zip if it's a lot of files.
There is ffmpeg for Windows.
That helps me exactly zero when I'm running something that is compiled for Linux and has no context in which to use the windows version. Which may not even be compiled with the same ./configure settings and would therefor potentially be missing entire codecs available to it.
This ladies and gentleman is the problem with discussing Windows design patterns on the internets. They say "use this" and you say "well it's broken in X, Y, Z" ways and instead of fixing it they say "you're using it wrong". No. Maybe it's just an inferior architecture.
This ladies and gentleman is the problem with discussing Linux distros on the internets. They say "install this" and you say "well it's broken in X, Y, Z" ways and instead of fixing it they say "you're using the wrong distro". No. Maybe it's just an inferior desktop OS.
You are very far from the average WSL user, especially with that Gentoo mention.
Why are you using ffmpeg under wsl?
What's the point of this question? It doesn't matter.
The entire point of wsl is to be able to run code compiled for Linux on Windows.
ffmpeg underpins so many things these days. It could be used to extract frames in a PHP based website, convert something to a gif, or a demux an mkv. You may as well be asking me why I'm using a computer.
I am not very impressed with macos file system speed when using docker, same problem really.
If you build 100+ megabytes images several times per day it really sucks.
Same here, though I went to Linux first for several years. WSL file speeds, especially when running npm install, were the impetus that ultimately got me to switch off of Windows.
Sounds like you were just doing it wrong
Either you run npm install from Windows if you are operating on the Windows file system or you run it on WSL if you are operating on the WSL file system both cases will be very fast
My memory is murky here, but I don't think WSL even had two different file systems at the time.
How would you run Linux without Linux having its own file system that doesn't even make sense
The entire Windows operating system is doing it wrong. Seriously who daily drives windows these days. lol.
Well before Windows I spent years with both Linux and Mac and I found Windows to be a good mix of stability and suitability for development now that WSL is a thing. Also for gaming it's the best by a long shot so just all around I've found it to be best and WSL made me never miss Linux.
Nah my frames on Linux beat yours easily.
Unlikely due to the better and more stable NVIDIA drivers available to Windows and the greater compatibility with every game without having to mess around with configuration files or other hacks. But you do you.
Linux drivers are now first class and are faster and easier to install than any Windows drivers. There's no bullshit extras with them. They just work. Plus steam launches games in containers so there's zero configuration. If you don't know what you're talking about it is in fact better to say nothing than to just make shit up.
It's great that gaming on Linux has gotten a lot better over the last several years but let's not pretend like Windows still isn't far ahead on this
Also how can drivers be easier to install than on Windows when updating my GPU driver is one click?