I can haz smoller NixOS ISOs?
natkr.com107 points by logickkk1 7 days ago
107 points by logickkk1 7 days ago
What's most amazing is how easy it is to switch kernels on NixOS as this article touched on. This prompted me to switch to Cachy kernels, and after some caching delay (apparently need to switch once before caches set) I was able to take advantage of a completely different OS's core strength. Truly the one OS to rule them all.
Sorry, I am a bit confused about the caching delay, which seems to suggest you can switch kernels without rebooting? That's not what the wiki suggests[0], and what even happens to running programs?
Or do you mean just changing the "default" boot to a different kernel, which in other distros would require changing the boot loader config?
If memory serves the cachyos-nix flakes sets a cache for pre-built kernels, but that cache isn't available until you rebuild. So if you want to use the cache you need to do two steps, add cachyos to your inputs, rebuild, switch the kernel, rebuild.
Then reboot to use the new kernel.
The article does not mention switching kernels. It comments on the size of the kernel + modules; and trimming down unused modules.
Of all the things to get nixpilled over, this isn't one.
That's how I manage all of my virtual machines: building an ISO from a NixOS config and booting it as a virtual machine. I'm going to take some time to see what bits of this I can copy to slim down my ISOs.
One additional benefit: I build all my software from source (by disabling the nix cache) so stripping out these extra programs will not only slim down my ISOs but it will also reduce the build time.
You could manage those VMs with https://microvm-nix.github.io/microvm.nix/ that helps with mounting the nix store read only from the host into the vm. That should save more space than trying to reduce the dependencies of the system closure of the vm.
Curious bout your use case for building all software from source, is it because you're worried about the supply chain since nixpkgs builds don't have reproducibility guarantee?
I was already building the vast majority of it from source because I enable CPU optimizations for the specific microarchitecture in the machine (nixpkgs.hostPlatform.gcc.arch and nixpkgs.hostPlatform.gcc.tune), so once I learned about the risk of supply chain attack on the nix cache, disabling it entirely was a pretty small change.
So far, I'd say the biggest negative (aside from the build times that I was already experiencing due to the optimizations) is that GNU savannah will temporarily IP ban you when you download too much from them. For example, building the grub that is used for the ISOs downloads like 70+ patches from GNU Savannah which frequently triggers the IP ban.
I assume you like it more than Gentoo?
Building from stage1 with customized CFLAGS was all the rage then…
> the risk of supply chain attack on the nix cache
Do you not trust hydra, the infra hosts, the people, all of it? What could be done to improve the cache's posture?
One suggestion for improving the cache's posture: Give each dedicated builder machine a TPM. On each builder, generate a TPM-backed signing key. Only use those TPM-backed keys for signing packages that end up in the cache. That way we'd ensure that signed packages _must_ have been built on the dedicated build machines (due to the TPM) rather than potentially uploaded from a compromised developer machine using a compromised signing key.
It seems the current situation is there is a single signing key that has been in use since 2015 which implies that key is not tied to specific hardware and therefore could be leaked/exfiltrated/copied: https://github.com/NixOS/rfcs/pull/149/files#diff-ac4cc42774...
https://sidhion.com/blog/nixos_server_issues/
Here's another good article on the topic
It's unfortunate that Perl and Python are core deps, as well as Bash
Note that's somewhat out of date:
> `bin/switch-to-configuration` is a Perl script from the beginning
Since NixOS 24.11 the default is `switch-to-configuration-ng`, in Rust. That is a 2.8 MB binary, compared to NixOS's 55 MB Perl distribution. Thus such Perl-less systems shrunk that dependency by 20x regarding activation switching.
And since NixOS 25.11, `nixos-rebuild-ng` in Python replaces its former Perl counterpart, see https://wiki.nixos.org/wiki/Nixos-rebuild
There is a project by a group of nix/nixos enthusiasts aiming to replace some of these core deps https://github.com/manic-systems/nixos-core. I havent tire-kicked it much, but it seems useful.
Less than they used to be: https://github.com/NixOS/nixpkgs/issues/267982 https://github.com/NixOS/nixpkgs/issues/428908
> we should be able to disable shipping Nix entirely, by setting nix.enable = false.
It was at this point I began to question the entire exercise. If you don't want nix to even be installed, do you really want NixOS at all?
It would probably be much simpler to just build an image from scratch with the packages you want, composed in the way you want them, rather than contort the NixOS "UX" to produce the image you want.
Nah this kind of thing is pretty common for servers or little embedded computers. If you're always building the image on some other machine, there's no compelling reason to include that binary or it's dependency closure on the system.
Typically someone who goes for this indeed likes and runs "full fat" NixOS on some systems. What they want is to get really small images for some special purpose, like containers or disk images for a puny old Raspberry Pi, and build them using their usual NixOS workstation or server or whatever.
Well, if someone wants a less painful method just install and run buildroot instead (and if you enjoy pain use yocto and bitbake).
For Nixers, this isn't painful; it's magical! :'D
One of the great joys of NixOS is that configuration of existing features is continuous with development of new features. Both are just NixOS modules. In the same way, configuring your desktop is continuous with configuring a server is continuous with preparing a netboot image for an embedded device. If you know how to configure a NixOS desktop, you also know how to prepare your own custom NixOS install media. If you want it to, Nix can cure the stomach ache HCL gives you when you write Terraform.
Nixpkgs is also the largest collection of packages in existence.¹ Anything you could want to throw into an image is likely already there.
The author of the blog post is already a Nixpkgs/NixOS contributor, too, so they might also be thinking "some of the improvements I make in shrinking dependency closures or finding deeper ways to turn off certain features are potentially upstreamable".
And even if the experiment is a wash, in exploring this kind of (ab)use of NixOS for generating tiny images, a NixOS user is also learning more about the dependency relations and other structure of the NixOS systems they're already running, which might be interesting or rewarding.
I'm not saying that as a Nix enthusiast I'd never use Buildroot or Yocto, but I understand the appeal of exploring how far NixOS can be pushed for this and how easy or hard it is to do.
While we're here we should also note that NixOS isn't really the Nix tool for this job anyway. There's a cool project called not-os² where minimization is one of the goals that has already succeeded in getting the image size down to < 50 MB.
There's also a project targeting commodity wifi routers called Liminix, which likewise takes some NixOS ideas and uses them to produce teeny-tiny images, (presumably even smaller than not-os, given many of the target devices don't include so much as 50 MB of flash).
--
1: https://repology.org/repositories/statistics/nonunique
> build an image from scratch with the packages you want
Using what? Using NixOS to configure a system is orthogonal to the system actually running the Nix binary. Nix/Nixpkgs provide well maintained package derivations and module configuration for the largest amount of software of any ecosystem. IMO it's far simpler than Yocto or Buildroot or the dozen OCI container builder ecosystems that go in and out of favor.
Using Nix of course.
My point is that if you care deeply about what is being installed in the image, size, dependencies, bloat, etc. then perhaps using the NixOS abstraction is the wrong approach. Instead of building "down" by taking things away, build "up"
> My point is that if you care deeply about what is being installed in the image, size, dependencies, bloat, etc. then perhaps using the NixOS abstraction is the wrong approach. Instead of building "down" by taking things away, build "up"
Those aren't necessarily oppposing points.
NixOS is a declarative distro. It also happens to come with some defaults that, I assume, caters to the commonly expected use case (and maybe has some historical roots as well).
NixOS is not a minimal, build-from-scratch distro. It's more opinionated than e.g. Arch. For example, it ships with firewall turned on by default (https://nixos.wiki/wiki/Firewall). Another example: the default list of packages is somehow Perl, rsync, and strace (https://search.nixos.org/options?channel=26.05&query=default...). Blanking this default to an empty list is IME harmless.
The declarative nature is probably the subtext the author is trying to convey: what are the things one can do to disable these defaults, to reach a very minimal system (ISO really) that one can then build as one wishes.
Seems to me that such boolean flags do not force you to think about "building up" or "building down". You just declare what _is_. Who cares what the default is? When you're building a minimal thing, changing some defaults is fine.
As your sibling points out, it's not just about boolean flags: it's about the abstraction, which is opinionated.
I posit that it is perhaps more laborious to untangle those abstractions than to just write your own.
It's like improving the lighting in your house by going to the breaker panel and flipping breakers and splicing extension cables that you run up your stairs and down your halls.
I think NixOS itself could do a much better job here in terms of moving stuff from defaults into profiles.
But yes, it'd also be nice to have something closer to dockerTools.streamLayeredImage.
Yes? Because NixOS is configured by Nix code. But the user of the image itself doesn't need to run `nix`
I disagree, I can reuse my entire tool-chain that I develop on, easily package things, get the whole reproducibility, integration tests, blah blah, and make an image for maybe an embedded or net-booted image that I would prefer be smaller. Especially if I have no intention of tinkering or rebuilding things on the final prod target. Much more reproducible and easily portable than say a giant folder of alpha-numeric bash scripts, installation scripts, etc.
Pretty common method these days to build something just for the task at hand and consider deployment immutable requiring a new build of the ISO.
> build an image from scratch with the packages you want, composed in the way you want them
That’s exactly what this does, elegantly
Hey, thanks, nice adventure! I will have a look. I'm trying to ditch Arch for NixOS and I'm starting from Distrobox probably. Super useful!
Why ditch Arch?
Mainly because I favor NixOS's idea of control over the system, and because Arch is getting too popular (steamos, omarchy), making it a target. The recent AUR plague is actually a symptom, to me.
I got a bootable NixOS iso down to 91 MB. Pointed Claude at the Nixpkgs repo and asked it to strip things aggressively and inspect the build closure iteratively.
But the resulting ISO:
- has no network
- can't switch configurations
- doesn't have a text editor
https://gist.github.com/siraben/a8fce9912891d85e1ec3cf74081b...
I miss tomsrtbt and all the other single floppy distros, it was surprisingly complete for 1.44MB (including networking, with more protocols than the average browser, and text editors): https://web.archive.org/web/19990506100919/http://www.toms.n... (mulinux iirc boasted about having 100 commands or something like that)
I wonder if it makes sense to create distroless containers like this.
i'm doing this using nix2container and a rust program that generates a landlock filter so i can mount nix store and only the supplied derivates are visible :) coming to you in a few month, tix.im
tbh, this is the most powerful computer program I have ever seen. Working on this now for over 6 month, ~600k LOC. I think it will grow to 800k in the base package. Unfortunately, uncompromised security requires quite some LOC.
This works: tix workflow submit ../vx-maintainer/workflows/change-management/default.nix --follow -- --feature-branch="feat/2lines" --instructions="tui overview should show running flows in 2 lines. 2 line: running steps, done, failed, etc. reports counter: info/success/failed/warning/error sparkle line with load ... . then if extended by double click, each running step has a line. what step, memory, cpu usage, token counters, ... subgraphs should be rendered under the parent flow. Title should always show that subgraph. details has link to parent " --source-repo=/home/poelzi/Projects/tix/tixim --implementor=opencode\#gpt-5.5
I get a human gate after the plan is finished where I can give feedback (in this workflow). And then another gate when everything is done. 4 different models, multiple harnesses, multiple reviews, rest cases etc. :) It makes me so smile watching the workflow progress. And the shit becomes global in a few days...
"Can haz smol"
Why are we talking in fake childish speak?
Considering "I Can Has Cheezburger" and that kind of lolspeak is nearly 20 years old now, I'm not surprised to see some people unfamiliar with it nowadays :-)
It's always fun to see an interrobang in the wild.
[dupe]