Rewriting Bun in Rust
bun.com795 points by afturner 2 months ago
795 points by afturner 2 months ago
Article did a decent job of showing discipline and care and human involvement to assert the automated rewrite was done diligently, as best as it can be when using AI for it. I does make me feel a bit more comfortable about it.
As an aside, I don't know why anyone would not want to use a memory-safe (and possibly race-safe) language in 2026. Rust gives you that in a performant package, so if you are turned off by GCs and immutability for performance reasons, you still have the option to use Rust.
I can understand when you need the absolute best performance and you decide to drop to down to C++, and I also relate with just personal preference, but beyond those it seems a no brainer to me.
> As an aside, I don't know why anyone would not want to use a memory-safe (and possibly race-safe) language in 2026.
The rust compiler is very slow. The best way to speed it up appears to be organizing a codebase in many crates. This is not preferable ergonomics to many. Beside that, for many problems, a garbage collector eliminates a large amount of defects (including the ones stated in the article) without any added friction, whereas Rust asks that you think in terms of ownership. This is not preferable ergonomics to many.
I realize what I'm saying above, while true, doesn't give a clear example. Many gamedevs would rather iterate with a language that is lower friction, not only because game code is finnicky (like frontend UI code) but because the build process can be unique. Many gamedevs prefer to iterate with hot-reloading, and asking them to use a slower compiler is asking them to accept greater latency in that cycle.
I do not claim that these reasons apply to everyone.
Game engines are typically in two languages, one for the engine itself and one for scripting. That even goes for Unity: in Unity, C# is a significantly more powerful than average scripting language (for lack of a better term), but the engine itself is still C++.
That's not to say that you couldn't write a commercial game engine with something like C# that stands shoulder-to-shoulder with unity and unreal, but it doesn't seem like anyone has attempted to do so. Maybe it's the decompilation fear.
Also, it would continue to make sense to use a scripting language alongside Rust.
> That's not to say that you couldn't write a commercial game engine with something like C# that stands shoulder-to-shoulder with unity and unreal, but it doesn't seem like anyone has attempted to do so.
Stride should mostly fit the bill: https://github.com/stride3d/stride
Their homepage: https://www.stride3d.net/
Even the physics engine they use is in C#: https://github.com/bepu/bepuphysics2
Not a very popular game engine (never got the attention that the likes of Godot did), but it's nice to work with! It’s up there with Stride in regards to the “indie” game engines I like, maybe alongside jMonkeyEngine (since you typically don’t see that many Java game engines either).
> It’s up there with Stride
Edit: I mean that it's up there with Flax: https://flaxengine.com/ whoops
Stride, Flax and jMonkeyEngine are all pretty cool, not just Godot and O3DE!
As someone who has almost no familiarity with game engines, it seems the success of this port was largely possible due to a comprehensive test suite written in a runtime agnostic way. What might be the equivalent test suite implementation required to successfully port a game engine to another language?
Gosh, I don't think any game engines have particularly good test suites at all. GoldSource and Source are the only ones that I have any real experience with and neither seems to have anything (Source may have a handful of things but nothing approaching baseline let alone comprehensive).
I have no idea how game devs handle big refactors other than lots of manual testing.
> Gosh, I don't think any game engines have particularly good test suites at all.
What? I do understand that a CRUD app with little to no logic do not focus on test.
But any kind of engine.. i can not understand that it's not a priority? In general I'm not a big fan of unit test on simple websites, they do not give any real value compared to effort
But I remember working on a complex codename. It was extremely important to have close to 100% unit test in the core part. Saved me a lot.
Mostly, because a lot of game engines are ancient relics, tracing their lineage all the way back to Quake 1.
The development practices are not exactly up to date, and game development is in no hurry to change. It doesn't help that software development wages there are not at all competitive - game development selects for passion, not skill. People who want to build robust modern codebases and people who want to build AAA games are different people. So there aren't many game devs who want to push for better test coverage.
But it's also because game engines are dealing with many, many things that are hard to test for.
You know how messy it is to test a website for "does this layout look right" or "can you navigate from A to B"? Now multiply that by complex 3D geometry. A lot of what game engines do is dealing with complex 3D geometry, where the primary verification is "does it look right" and "does this interaction feel right". Which is why game development traditionally has wide human QA, and slim unit testing.
Only now do we have software that can sort of, semi-reliably, automate testing for "does it look right".
A game engine has a vast space of possible outputs, all of which will be considered good enough.
Its primary output is pixels and sound. Those are hard to test in a reasonable way. Screenshot testing is useless in a codebase where most of the changes are about making the pixels prettier.
> In general I'm not a big fan of unit test on simple websites, they do not give any real value compared to effort
And then your site gets more traction, features are added, things are refactored.
And now you're looking at what used to be a simple site, still with no tests.
It's a habit, that pays off long term. You don't do it for yourself, you do it for who changes the code next. Might be you, or not. Even when you do it yourself, you can't hold all the requirements in you brain, you write those up in tests
I agree - maybe some modern game engines have unit tests for stuff like their physics engine and their "standard library" code (optimised data structures, maths utilities, string utilities etc) but as others have said, testing sound and vision is hard, and something that hasn't been mentioned is all the different flags in a game engine - testing even a sensible subset of all the different sound and rendering variables would be a huge undertaking.
One option would be to have an input replay alongside captured outputs (audio visual), at some fixed framerate. Capturing intermediates (scene graph etc.) would probably also be valuable, as that could help nail down why something is failing.
Or you could do it [as I recall the project being called] the scientist way. You still have the old code, so you could replay inputs against each and compare. Probably more realistic because uncompressed video would be a ridiculously huge dataset. This would be more resilient in the face of testing hardware and driver drift.
Historically game engines are the worst offenders when it comes to unit testing. I'm not sure if that's still the case - but that's why I erred on the side of integration tests.
Box3D just showcased some stuff including deterministic replays. If you wanted to port that, you could probably import the replay and make sure it plays back the same way in your new language. I think it captures the inputs and forces applied, not the pixels.
I suppose rendering is a component of a game engine too though, not just physics. I don't know how to do that reliably. Even if you captured pixels, it'd be annoying. If you've ever tried doing screenshot based diffing on web you will know that slight changes in aliasing in Chrome bugger everything up. Things that should be equivalent randomly aren't but not in a way that any human would care.
UE 5 doesn’t come with a scripting language even though there is blueprints which a node editor easy enough to be used by non dev. Studios like embark (The Finals, ARC Raiders) have been successful integrated AngelScript in UE 5 and use it in these games. UE 6 will see the appearance of Verse: the scripting language used for Fortnite (and the end of blueprint at some point). At first I was sad to see yet another language but it is worth looking at the doc to see why and how it makes a lot of sense.
> The best way to speed it up appears to be organizing a codebase in many crates.
A "crate" in Rust is the unit of compilation. In C, a file is the unit of compilation. Rust just lets you have a compilation unit that's composed of more than one file (without having to resort to C-style textual inclusion). But if you want, you can certainly have one-file-per-crate, just like you would in C. And what's nice about having many crates is that crates forbid circular dependencies, which trivially enables coarse-grained parallelism in the build system. So yes, organizing a large codebase into crates is the best way to achieve parallelism, but that isn't something to be deplored (and strictly controlling circular dependencies is useful for comprehending large codebases in general).
The forbidding of circular dependencies is exactly what makes it hard to achieve parallelism! It means you have to draw nice clean module boundaries and split your compilation units there. Clean boundaries sound nice, except… what if the module is getting large? Can you just take half the module, ctrl-x, ctrl-v into a new file, and get faster compilation times without having to do any massive refactors?
In C, usually yes.
In C++, sometimes yes. It depends on how template-heavy the code is, but if you have some discipline you can keep most logic out of headers and thus easily splittable.
In Rust, almost always no, because of circular dependencies. You can try to work around it by adding `dyn Trait` everywhere, but that requires a lot of code changes and comes at big ergonomic costs (and a small runtime cost).
Which is why in practice, Rust compilation units are almost always larger than C++ or C compilation units. Rust can sometimes be competitive with C++ on compilation speed anyway, thanks to a smarter build system and not having to re-parse headers a billion times, but usually it's slower.
> Can you just take half the module, ctrl-x, ctrl-v into a new file, and get faster compilation times without having to do any massive refactors?…In Rust, almost always no, because of circular dependencies
This feels like a strange, overly-specific complaint. It reads a bit like “When I write entangled code, it’s hard to untangle”. Like, yeah, the only thing that’ll save you from that is…not writing entangled code? I’m not of the opinion that the argument of “yeah but C lets me do whacky stuff” is a particularly strong line.
FWIW, letting a module grow, and then splitting modules up by cut-and-pasting stuff out along natural domain lines generally _is_ how I write Rust. Largely due to how easy it makes it to construct modules and submodules.
I addressed some of these points in my reply to a sibling comment:
https://news.ycombinator.com/item?id=48856158
I may have been a little overly specific, since there are other issues besides cycles that also block splitting crates apart, but from what I’ve seen it’s very common for modules within a crate to have cyclic dependencies and therefore not be easily factorable into separate crates.
The industry accepted way of handling circular dependencies is to not have them and heavily lint against them in languages which permit them in compilation or runtime.
You’re thinking at the wrong scale. Rust allows circular dependencies just fine, within modules in a crate. And it’s extremely common for modules within a crate to have at least some circular dependencies - type X has some method (trait impl of inherent impl) that mentions type Y, and type Y has some method that mentions type X. In fact, I would be surprised if you could name me a single medium-size-or-larger popular crate that doesn’t have at least one cyclic reference between modules! Though, sometimes those cycles are not essential and could be avoided by splitting up modules. But sometimes they are. Either way, in C or C++ those modules would probably be their own compilation units.
That said, you also run into related parallelism blockers without cycles. For example, the orphan rules force most trait impls for a type to be in the same crate as the type definition. Also, a module which has no source-level cycles will often have cycles after monomorphization. In this case, Rust doesn’t prevent you from splitting the code into crates like it does with source-level cycles, but you do lose most of the actual codegen parallelization unless you can switch from generics to trait objects.
> The rust compiler is very slow.
It's not “very slow”, that's a tired meme. It's slower than it could/should, but complaining about rustc being “very slow” is a clear misrepresentation, especially when everybody seems to have been fine with tsc's historical performance for instance. It could be nice if it was faster indeed, but people claiming it's “very slow” are just showing they never worked with it.
> The best way to speed it up appears to be organizing a codebase in many crates. This is not preferable ergonomics to many.
In this context (where you don't plan on publishing you stuff on crates.io) a “crate” are just a directory at the root of your repo, the ergonomic impact is literally zero.
I am using Rust since 2016 productively... before I dabbled with it.
Rust is not compiling any slower than a comparable C++ codebase for me.
It is compiling much slower than a managed language like Kotlin, C# etc. though. Which is an unfair comparison anyway.
The problem with e.g. gamedev is the iteration cycles. It's very creative work and not so much your average engineering job (apart form game engine dev).
But again, that's an unfair comparison, because even in Unity and many if not most other game engines, there is a scripting language that is used for most of the game logic, that doesn't need hyper performance...
Still, when compared to such languages, Rust indeed does compile slowly AND one has to say: Rust is selling itself not only as a safe, blazingly fast systems programming language these days, but it also is used for its type system etc. and then, when you come from a managed language, compile times really suck.
(Although debug builds are better, they are not an option for game dev for example, because the binary is just too slow then... 200fps vs. 20 :X)
Why do you think it is not slow? As far as I know the only language that compiles slower is C++, and even then the compilation speeds between c++ and rust seem to be comparable. I believe c, Fortran, zig, C#, Java and golang are all faster compiling languages. That makes rust pretty slow in my book. I get that it doesn’t bother everyone, but that doesn’t change the facts.
> Why do you think it is not slow?
The average cargo check for the projects I've worked on, usually finish in less than 1 second, with `cargo build` completing in a single digit second (often below 2s), it's not slow by any means.
> I believe c, Fortran, zig, C#, Java and golang are all faster compiling languages.
Sure, but the difference between type checking is 10ms and type checking in 500ms is barely noticeable for a human being anyway, despite the x50 difference.
> That makes rust pretty slow in my book.
“Slow” is a perceptual thing. It doesn't matter if it's slower in absolute benchmark performance. If it doesn't slow you down in your work it's not “slow”.
> As far as I know the only language that compiles slower is C++
Typescript's compiler is much slower than Rust's, but it's plenty fast enough for most people and you almost never see complains about it because it mostly doesn't matter outside of pissing contests.
Typescript's compiler is much slower than Rust's, but it's plenty fast enough for most people and you almost never see complains about it because it mostly doesn't matter
But you have to compile Rust code to run it. You can run TypeScript code without type-checking it. That’s a massive difference in the development workflow.
The new TSC, supposedly 10x faster, will be very pleasant to have but not as much of a game-changer as you might expect. A 10x faster Rust compiler would be incredible.
> But you have to compile Rust code to run it. You can run TypeScript code without type-checking it. That’s a massive difference in the development workflow.
And yet I'm waiting for TSC every day while almost never thinking about rustc…
> The new TSC, supposedly 10x faster, will be very pleasant to have but not as much of a game-changer as you might expect.
It will be very nice, but I don't expect it to be a game changer, tsc isn't fast but it's fast enough to get the work done, the annoyance is there but it's objectively minimal. Anything else is pointless internet language war.
> A 10x faster Rust compiler would be incredible.
For development? Not really, not for me at least. Against the endless rants about rustc's performance on HN, absolutely.
> tsc isn't fast but it's fast enough to get the work done, the annoyance is there but it's objectively minimal.
For our small codebases, maybe. The tsc-in-Go announcement had VS Code compilation go from >120 seconds to ten.
"Compiling" is vague here; for TS I think you need to distinguish bundling from type-checking.
If you're talking about bundling the entire app, 120 seconds down to 10 seconds is a great saving, but other bundlers can do it much faster already. Most people don't bother using TSC to bundle their code. I can understand why Microsoft might insist on using it, but it's not typical.
For type-checking the entire app, I would see that as akin to running all the unit tests. 120 -> 10 seconds is excellent, but not something that should be a constant bottleneck in your development workflow.
I can definitely see that 10x improvement being crucial on large teams (and/or LLM swarms) where the speed of the CI queue becomes a bottleneck.
Exactly: tsc was “slow enough to be painful” in the context of the 2.5Mloc codebase of VScode.
But it's not too slow for most people.
Likewise, Rust may be slow enough to be painful for some big projects that need to often rebuild from-scratch in release mode, but that niche is definitely much smaller than the size of the “rust compiles too slowly” crowd on HN.
I gave Rust classes in university a few years back, and literally none of my students complained about slow build times. At this point it's much more of an internet meme than an actual pain point for most devs.