Using bubblewrap to add sandboxing to NetBSD
blog.netbsd.org96 points by jaypatelani a day ago
96 points by jaypatelani a day ago
Bubblewrap is a really interesting project, really worth checking out.
https://github.com/containers/bubblewrap
It's the base for Flatpack, the thing that makes Flatpack be sandboxed.
I use it to run Claude Code / Codex / Gemini CLI, to make sure that they have a limited / fake view of my system.
You can bind directories into it, or overlay them into it, restrict other kinds of access.
If Docker is a thing in a scale between a VM and your OS, Bubblewrap is a thing a scale between Docker and your OS. You use your OS, instead of installing and managing an OS like you do with Docker, but you get filesystem and process isolation like with Docker.
Though I had an issue where I cannot use `--new-session`, which is kind of dangerous to not use, but you can get around it if you use seccomp to block ioctl calls, and ptrace.
A question.
How do you know what permissions are required by an application, to write a bubblewrap script?
In AppArmor, you exercise the application and aa-logprof suggests permissions requested by application. If you know AppArmor, usually you can refine those suggestions and write a profile. It may not be ideal, as aa-logprof’s permissions are multiple choice suggestions, require user knowledge and may be too broad or specific, but it could work. You will see that there are many and all kinds of permissions, and there is no way that you will be able to guess them without aa-logprof.
What is the equivalent of aa-logprof in bubblewrap and how do you find the required permissions?
I documented what I came up with for Claude Code: https://blog.gpkb.org/posts/ai-agent-sandbox/ However, I couldn't get this to work for Codex, it kept failing at the auth bit and I couldn't figure out how to fix it. Anyone got a working solution for Codex?
bubblewrap escapes are not unheard of. Infact, it's a common theme that the general linux landscape lacks strong sandboxing, even if you use bwrap, firejail, etc. Especially linux desktop, a security firehazard to say the least unless you are using QubesOS
As a side note NetBSD is THE ONLY operating system of which binaries are fully bit by bit reproducible.
Moreover it vendores patched gcc compiler, so the entire toolchain is reproducible too as well.
How cool is that?! For the record golang brings it further - its entire package registry containing +40mln packages is bit by bit reproducible.
Are you sure GNU Guix System doesn't share this attribute?
Unfortunately it doesn't - even if our package definition doesn’t change output may, since software referenced by the package’s build function is not bit by bit reproducible.
The biggest issue is with Linux Kernel itself. Once kernel reproducibility will be handled, we can practically claim Guix, NixOS, Debian or any other Linux distro is reproducible as well, since with little effort we can avoid installing packages which will colour our reproducibility.
For desktop environments it’s a bit worse, but there are huge efforts to make this happen for all packages in package registries of aforementioned distros.
Can you explain what reproducible mean in this context? Does that mean that you can recompile everything from "scratch" or does it have a deeper meaning?
Reproducible builds ensure that you can build the same binaries with the same source code. Nothing like the current date for instance gets in the way of getting a different build.
This allows independent people to check that provided binaries don't contain malicious stuff for instance. Ultimately, it lets you download binaries instead of rebuilding everything yourself locally if the binaries have been independently reproduced.
You might be interested in checking:
The provided binaries may still contain malicious code but it guarantees that no malicious code has been inserted in between the build process of the published code. So if your binaries contain malicious code, you can be sure that all other users of the software version are affected, too.
does anyone practice dual build pipeline? eg: 1 by your devops team and another one by your security team and compare binaries hash later. To verify everything is reproducible.
is it a common practice?
It is not common outside of security inclined communities like cryptocurrencies. It should be and we are slowly moving there.
It's possible to run a full DE bwrapped.
bwrap --dev-bind /dev /dev --tmpfs /tmp -- labwc
bwrap --dev-bind /dev /dev --tmpfs /tmp -- kwin_wayland konsole
bwrap --dev-bind /dev /dev --tmpfs /tmp -- startplasma-wayland
That's obviously super permissive, but from there can take things away to expose as much or little of the host system as needed.
For my system I'm working on making a few bwrapped "zones". E.g. start a terminal that can see a certain subset of files, and can configure whether it should use gpu or not, have internet or not, have access to local network or not, etc. A bit more project/environment focused than bwrapping programs one by one.
Always nice to see NetBSD posts here, that nice OS gets hardly any press.
I experimented bubblewrap as a better alternative to firejail. Unfortunately there are scripts for few applications. It’s tool for developers. Users are better off flatpaks powered by bubblewrap .
I have been using firejail for a long time now. It seem to work well for me. At times I have to write my own profiles, yeah.
The situation with firejail is different. There are a lot of profiles, and they are easier to customize and write.
But firejail needs root privilege, so there are arguments that it may actually reduce the security, because even though it sandboxes some applications, non-privilege processes may gain privilege if there is a flaw in the fire jail itself.
Firejail is also criticized for complexity.
Firejail does not need root privileges at all. You are probably referring to SUID. It is a "setuid ELF 64-bit" executable. I run it with my own user only.
Complexity? It just supports a lot of stuff. See the manual page. It seems simple enough to me.
Yeah, I meant the same thing! If you run it under your non-root user, and there is a vulnerability in firejail, the jailed process could run as root, right?
Complexity referred to large code base and attack surface.
I haven’t tried it myself. I would be interested knowing, whether it might lower the security in some cases. Maybe it’s useful to sandbox apps that don’t have flatpaks.
SUID by itself is not a security liability. Typically, programs using SUID quickly drop their additional privileges during startup to avoid the scenario you're describing. So the vulnerability in firejail would have to be early during startup before any of that.
Still, it can happen of course, particularly with memory-unsafe languages. Conversely without root privileges you have to sacrifice strength of your sandbox, widening the vulnerability window in the app you were trying to constrain to begin with.
Meanwhile bubblewrap relies on user namespaces, which come with their own set of security problems. They are disabled by default in many distributions for that reason.
Yeah, it runs with escalated privs. People wager that SUID binaries are extremely risky and this seems totally reasonable to me! I don't know where to find a really good security analysis of the risks, however. Firejail is most likely not bullet-proof, although not soft by any stretch of the imagination and I don't know how often and how many security researchers are profiling this tool.
My gut sense is that flatpak gets much more scrutiny since it ships, and firejail is typically not shipped, but another package as far as I can tell (maybe in some specialized distros?).
I would have loved to see something like OCI containers.
This would underpin that, right? First you build the namespace primitives, then you put the handy wrapper on top that composes a root fs and spawns a container from it using those namespace primitives