Linux from Scratch
linuxfromscratch.org306 points by Alupis 4 hours ago
306 points by Alupis 4 hours ago
This was probably the best thing I've done to learn the ins and outs of a running Linux system: I think it would be amazing to re-do it with a modern Linux stack (systemd + Wayland), but it can really remove all the "magic" from the full OS implementation for you.
However, I've done it in 1999 and ran that system until 2001 when it became too much of a trouble to recompile everything and battle dependencies manually — note that LFS was not as detailed then either, so many dependencies you had to track yourself, and some were very obscure!
Yes, the time investment was high, but I was a high school student starting college with too much interest in something like this and obviously enough time on my hands (after which I was so "smart" to switch to Slackware, a one-man show where I also ended up having manually compiled versions of "small packages" like XFree86 and GNOME, which I was contributing to: when "GARNOME" showed up, that was a revelation! etc)
So if you can afford it, do it: using Linux will never be the same again!
Wayland is quite simple actually. Nowhere near as much to configure compared to systemd.
Even systemd can be kept simple - I did this on manjaro.
Either way the power of LFS/BLFS is to adjust the system to your use case.
> it can really remove all the "magic" from the full OS implementation for you.
To some extent. Many things are missing IMO, in particular if you go to BLFS. But for the most part I agree with you - it is great that we have it.
We should extend it though.
> until 2001 when it became too much of a trouble to recompile everything
I use ruby scripts for that, tracking almost 4000 projects. Once gem-coop offers us an alternative to corporate-controlled rubygems.org, my main ruby projects will be republished (I retired in 2024 when RubyCentral tried to force everyone to cater to the new corporate rules as well as disown gem owners after 100.000 downloads).
I actually tried this back around 1999 as well.
At the time, it felt like I was just being told "type this" and "type that", with very little explanation of what anything meant or why it was done one way instead of another.
Maybe after all these years it’s worth giving it another shot? Does it work on a VM?
> Maybe after all these years it’s worth giving it another shot? Does it work on a VM?
I think it works in a VM just fine. I would recommend having some scripts to aid with compilation though. Even simple python scripts should suffice and probably others already did so. For the most part everything compiles well; if you have a few scripts that automate some parts, you could do this on a single weekend or two. It is quite straightforward, the explanations can be copy/pasted for the most part. One should know the basics of *nix very well though.
Last time I did it in around 2020 the reasoning behind every package, and the meaning of most compilation flags was explained. It was a good experience. Yes it works in a VM. A tip is to create regular clones as checkpoints if you fuck something up along the way.
I did LFS on hardware for advanced operating systems in college. After messing up an early step and having to torch it midway and start over, I made the entire LFS build directory into a local git repo. It was not the best use of git and there are better tools, but it did allow me to revert a mistake later and saved me time. So I call it a success.
They do offer a systemd version[1], along with a variety of other versions, including Gaming Linux From Scratch (X11 and Wayland), and Automated Linux From Scratch (with a build system)[2]
[1] https://www.linuxfromscratch.org/lfs/view/stable-systemd/
Agreed, I also did it in high school, circa 2005, and it was a fantastic experience. Learning how to build dozens of different projects and see how they all fit together really set me up for my career doing systems programming. I never actually used the system I built (I'm not even sure I got to a graphical desktop) but it was still a fantastic use of a handful of evenings.
> note that LFS was not as detailed then either, so many dependencies you had to track yourself, and some were very obscure
Why didn’t you just use Chat GPT to get your answers? /s
In all seriousness, I think the recent growth of Linux has been because information has become so accessible. Instead of having to peruse years of newsgroups posts, you can just ask a question and get an answer (that you should really fact check).
Quite a few comments refer to BLFS which is Beyond Linux from scratch https://www.linuxfromscratch.org/blfs/view/stable-systemd/in...
Every time I see this, I upvote it
I’m sure it’s different than it was when I was a teenager but building Linux from scratch was the thing that got me into computers as a kid
It shows that computers can be accessible _and modifiable_ at the lowest levels
Having done it as a teenager as well when it showed up in 1999 myself, that's probably the sweet point when we are smart (and persistent) enough to figure problems out, but also have enough time to see it through! :D
It is a bit different indeed - more things to compile nowadays. Things such as LLVM take quite some time to compile too. cmake and meson is also needed these days.
Other than that it still works fairly well.
It's been years since I went through this, but whenever someone asks me what they should read to get a deeper understanding of what a Linux distribution is, I point them to this.
Yup, it's where I got a lot of my linux knowledge.
I think that Gentoo or even Arch would provide pretty close to the same education level, though, with a lot less time to install.
Having installed Arch myself a couple of times, i think i would disagree. Not really much in that process that teaches you how linux actually works. It's more just about managing disk partitions and moving files around than anything else.
LFS is just on a whole different level, and is on my bucket list to complete the entire process one day.
I've completed it along with BLFS and I just don't really agree.
Like, yes you get pretty familiar with autotools, sed, and patch. However, a lot of LFS is in fact just managing disk partitions and moving files around.
LFS also glosses over a lot of pretty important parts like kernel configuration.
The docs from both Gentoo and Arch, on the other hand, are much more complete and practical in explaining things and also troubleshooting problems. And at the end of the process you're left with a system that can be easily maintained.
LFS is harder, but that doesn't really mean you end up learning more. Especially since it's pretty easy to lose focus and just rely on copy/pasting the next command to run.
Edit: Just an example of what I mean.
Here is the LFS discussion of filesystems.
https://www.linuxfromscratch.org/lfs/view/stable/chapter02/c...
And here is the same Gentoo discussion.
https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Dis...
Gentoo I understand but Arch? Does Arch go into compilation that much?
Not so much compilation, but it does delve into system management in a way that other OSes don't. Arch has few defaults setup for the user, so if you do it from scratch you'll end up needing to go through several of the general setup recommendations [1].
That's where you end up learning a lot about linux which is particularly practical. Other Linux distros, especially for the desktop, hide a lot of this information behind nice guis.
[1] https://wiki.archlinux.org/title/General_recommendations
Agreed!
As an addendum, you have to do it for your actual working computer, otherwise, doing it on a VM or a machine you don't use, you won't be learning nearly as much as there is no pressure to make it truly work for you (this is where learning happens, when the thing you wanted to configure, and LFS docs or web docs are out of date on, so you have to dig deeper).
agreed. I haven't done LFS, but ive done arch and plently of other distros for a good while and I definitely wouldn't say I have a rock solid understanding of the fundamentals.
> I think that Gentoo or even Arch would provide pretty close to the same education level, though, with a lot less time to install.
it only takes three commands to install Gentoo
cfdisk /dev/hda && mkfs.xfs /dev/hda1 && mount /dev/hda1 /mnt/gentoo/ && chroot /mnt/gentoo/ && env-update && . /etc/profile && emerge sync && cd /usr/portage && scripts/bootsrap.sh && emerge system && emerge vim && vi /etc/fstab && emerge gentoo-dev-sources && cd /usr/src/linux && make menuconfig && make install modules_install && emerge gnome mozilla-firefox openoffice && emerge grub && cp /boot/grub/grub.conf.sample /boot/grub/grub.conf && vi /boot/grub/grub.conf && grub && init 6
that's the first one
https://web.archive.org/web/20230601013339/http://bash.org/?...
I remember playing with Gentoo back in 2004-2005, going through the installation procedure from "stage 1" all the way through to the working system [1]
It looks like nowadays the handbook says just go from stage 3, which makes sense - compiling everything was kinda stupid :D
[1] https://web.archive.org/web/20041013055338/http://www.gentoo...
Other point is long time maintainability as well.. Like unistalling stuff you don't need etc. Or LFS solves it?
Yeah, that was a real lesson for me when I did LFS.
It was super neat when I got it running for a while, but young me that did it really didn't understand the concept of "Ok, but now you need to upgrade things". That was some of my first experiences with the pain of a glibc update and going "ohhh, that's why people don't run these sorts of systems".
I used versioned AppDirs for that, e. g. /Programs/Python/3.13/. If I don't need it anymore, the directory is removed and a script runs. Similar to GoboLinux. I do however had not use GoboLinux right now; GoboLinux unfortunately lacks documentation, LFS/BLFS has better documentation. Finding information these days is hard - google search has become sooooo bad ...
I'm still thinking that LFS taught me more about sed, gcc CFLAGS and bootstrapping than the underlying OS sadly
I bought the dead tree version of this book back in ~2006 or so.
I used to work in a business park in Seattle and the company next door to us operated a PC recycling warehouse. There were piles of old 386/486-era PCs in various states of disrepair just piled up behind their building. I'd go out there once in awhile and pick-through their piles looking for Intel CPUs, sticks of RAM, and hard drives. Find a good chassis with intact motherboard.
I loved putting that stuff together and installing Linux on those machines. I cut my teeth on Linux and LFS building computers out of those Frankensteins.
From "20 Years of Gentoo" (https://blog.nawaz.org/posts/2023/May/20-years-of-gentoo/):
"Even more common: “Oh, I’m not going to use Gentoo. I want to go all the way and use LFS!”
They never heed my warnings about it. Every one of them either quits in the middle of the install, or soon after, and swears off source based distributions for life.
Slackware and LFS are the Haskells of the Linux distribution world. People jump to the extreme end of the spectrum, and either get burnt or remain unproductive for life, when they should have just used OCaml or F# instead."> Slackware and LFS are the Haskells of the Linux distribution world.
Haskell is hard. Both Slackware and LFS are simple. I don't see the comparison.
LFS is even better in that it provides a ton of documentation. Slackware unfortunately lost out to the modern world. But heroic effort by Patrick.
> Haskell is hard. Both Slackware and LFS are simple. I don't see the comparison.
LFS/BLFS is hard to maintain as a primary OS. Gentoo is practical and easy in comparison. How many people do you know stick to LFS?
The point isn't so much about hard/simple, but jumping to the extreme instead of the pragmatic approach.
> Slackware unfortunately lost out to the modern world. But heroic effort by Patrick.
I dunno man, a lot of people still use it! Been on it every day about 4 years now, there's a modern ARM port in development, so it might not end up going away for a long time.
That Slackware is difficult to use on the level of Gentoo or LFS I think is mostly a meme and an overstatement; it's just very old-school. It has a nice installer and a good wiki.
Many of the gripes that I hear of managing other linux systems I seldom or never have experienced on Slackware. It doesn't get in your way or flippantly change things from one release to the next. It's a rock-solid choice.
The funny thing is that a lot of us older people started with Slackware back in the late 1990's, only because early Debian and Redhat builds on the Infomagic CD's were too broken to install or run reliably. Slackware 2.0 was pretty rock solid by comparison and installed out of the box on most PC hardware I could find at the time.
I got a partial burn, in that I realized how a working OS is an alignment of planets and if you fiddle with the physics wrong you get all kinds of magical phenomenon
My network stack was partially working depending on the program which initiated the TCP connection.. never again :)
As someone who has been using Linux as a daily driver for 25+ years and also used linuxfromscratch.org for building some packages, I would say, don't waste you time building from scratch. There is very little utility unless you are maintaining some arcane system professionally.
Stick to RPM based systems as dnf supports transactions. The ability to look at history of package installation and rollback to a known state solves most admin issues.
You're looking at it incorrectly. LFS is an exercise in learning for the sake of it, and therefore, not a waste of time. This isn't intended to be easy, but to expose and teach you the lowest levels of creating a functioning install.
I don't think anyone is suggesting one build Linux from scratch and then use it as their primary OS.
The value of LFS is not in having the system you build, it's in understanding it. After you've read and worked through the book, you've managed to produce a functioning GNU/Linux OS, and presumably you know what all the parts are.
From there, understanding any published distribution is a matter of understanding what makes it unique, maybe a different package manager or init system, or different userland packages. Regardless, the fundamentals still stand, and your ownership of the system is improved by having worked through the book.
> Stick to RPM based systems as dnf supports transactions. The ability to look at history of package installation and rollback to a known state solves most admin issues.
There are many ways to do this without RPM too. I used versioned AppDirs. NixOS uses hashed directories names and nix for description of states that are guaranteed to work. No need to have to cater to RPM.
Thank heavens I have you to stop me from trying to step out of my comfort zone.
I completed my Cross-Linux From Scratch distro in 2014, targeting the original Raspberry Pi since I was frustrated with the lack of (at the time) minimalist distros.
It was extra-hard, due to the cross-compiling nature of targeting the ARMv6 cpu family - but I learned a massive amount along the way.
Even though CentOS-minimal was released for Raspberry Pi by time I completed the project, I had so much fun it didn't matter. I ended up making a custom build system, consisting of a hodgepodge of bash scripts all wrapped together with a Makefile. My self-hosted Jenkins build server (old mini computer shoved on my book case) would run builds and produce the image artifact - those were the days...
The final distro image was ~40MB, which was impressive to me on it's own.
LFS/BLFS is great. I am not saying all of it is perfect; some parts could and should be more extensive. I am having issues with compiling a new kernel from scratch (so many options ... what do I need) and some options related to video cards and what not. But by and large, this is an example of what makes linux great: knowledge and application of knowledge. You rarely see this in other operating systems - definitely not windows really but also mostly not in other operating systems. Anyone knows the BSD version of LFS? Well ...
https://www.linuxfromscratch.org/lfs/view/stable/chapter08/g...
> the test suite for Glibc is considered critical. Do not skip it under any circumstance.
> Generally a few tests do not pass. The test failures listed below are usually safe to ignore.
I felt a bit uneasy writing something similar into my software e2e test suite, but hey if glibc can do it, why not!
> It's imperative to strictly follow these steps above unless you completely understand what you are doing. Any unexpected deviation may render the system completely unusable. YOU ARE WARNED.
Is this the Dark Souls of linux distros?
> Is this the Dark Souls of linux distros?
haha Yes. That's a bit what it feels like.
Had to go through it 4 times to reach a stable distro, I learned so much doing it
I haven't gone through this but now I really want to.
Moved to native Linux on the desktop a few weeks ago after 15+ years of using Linux on the server and spending a majority of my time in WSL in Windows for the last decade.
I've learned so many new things in this short period of time. Tracing down memory leaks with GPU processes, misbehaving GPU drivers, power saving modes, etc..
This is such an awesome project.
I had a lot of fun doing LFS plus a bit of BLFS and then I adapted it to my memory safe linux project https://fil-c.org/pizlix
This was a pivotal project for me as a young lad learning Linux and software engineering back in the day. Can't recommend it enough. So many little frustrations and painpoints to overcome, wasn't easy , and shows you the ropes of what's to come.
This and "Ray tracing in one week-end" are HN classics that I wish I had completed long ago.
I did this in 2001 on a 200MHz Pentium with 128MB RAM. Took about eight hours. Great experience.
I understand it still takes about eight hours. Faster CPUs but busier software cancelled each other out.
Some things never change.
> I understand it still takes about eight hours. Faster CPUs but busier software cancelled each other out.
> Some things never change
Yeah. Mostly the software stack is now much bigger.
Getting all of LLVM mesa linux cmake ninja glibc gcc binutils xorg KDE to work is more work than in 2005 or 2010. Also GNU configure feels as if nobody maintains it anymore. I still keep libtool around for legacy reasons but I don't like that.
Linux from Scratch is great for building a deeper understanding of how the different parts in linux systems work together at their foundation. Working through it also made me appreciate what distributions and package managers actually provide to me even more. However, i do often read people stating it helped them to "understand linux". Not sure what that means to be frank. Because you do not learn much about actually doing things with or on linux. You learn, essentially, how to knock together a linux system from the various components, the core part being, setting up a compiler (if i remember correctly, you rebuild gcc three times in the process to get an "untainted" compiler), i.e. build something like a distro pre-cursor if you will. These are great skills to have, but also very specific ones. They aren't helping you much in your day to day use of linux. I think everyone serious about linux should do LFS at least once in their career (and, contrary to some popular statements, you actually can do it quite early on, if you can read and follow a manual), just maybe don't have false expectations about what it will actually be teaching you.
Yeah I've gone through Linux from Scratch twice, but at some point I found myself just copy-pasting and to be honest I've never really understood how one would go from here to a modern distro (besides compiling a helluva lot more software).
> understood how one would go from here to a modern distro
Well, after LFS you go to BLFS. Compiling KDE isn't that hard if you use scripts that help you. The big problem I see is that a lot of information is missing. People need to know a whole lot. But if you managed to compile it, a simple "startx" should work fine. I even got KDE plasma to work on wayland. Wayland gives me fewer options than xorg though, so I went back to xorg.
Is it educational to do this on a VM or should I break out my old thinkpad?
In my opinion having it on a separate computer is easier, but you can also run this in one or three KDE konsole tabs, for instance, on an external hdd/sdd.
Both. Whatever works for you I'd say. I targeted the Raspberry Pi (Cross-Linux From Scratch variant), and a fake root (via chroot) and qemu. This was circa 2014 though.
These days the ARM64 processor on the Raspberry Pi 5 is probably fast enough to just build natively on it, no cross-compilation necessary. Cross-compiling adds a metric ton of complexity.
Having done this way back when on both: go with a VM first.
Targeting a known set of virtual devices makes a lot of things much easier when building LFS. Dev ux is also much nicer:, you get faster restarts, a socket and optional snapshots to go back to a known less broken state.
I wanted to set this up but then the thought of all the useful dbus stuff and device stuff and just all the nicities I take for granted with a mainstream OS … it seemed too daunting and I bailed. I’m 39 still haven’t done it yet and I saw this project maybe 20 years ago or something like that.
In the next part, I hope I can write a kernel from scratch, haha.
Grab Tannenbaum's "Operating Systems, Design and Implementation" - aka "The Minix book" - you will LOVE the book, and walk away having an genuine understanding of what's going on under the hood in a kernel, and be able to write your own
Amazon link https://www.amazon.com/Operating-Systems-Design-Implementati...
Years ago, installing Gentoo from an early stage was also a good experiment.
Nowadays they've deprecated all stages but stage3. It's still fun, but bootstrapping Gentoo from stage1 was a Linux-from-scratch-like experience (not quite, but similar).
One of the most incredible computer experiences.
Related. Others?
Linux from Scratch - https://news.ycombinator.com/item?id=41747966 - Oct 2024 (159 comments)
Beyond Linux from Scratch - https://news.ycombinator.com/item?id=39547118 - Feb 2024 (17 comments)
Linux from Scratch Version 12.0 - https://news.ycombinator.com/item?id=37648808 - Sept 2023 (28 comments)
Linux from Scratch - https://news.ycombinator.com/item?id=33734685 - Nov 2022 (9 comments)
Linux from Scratch - https://news.ycombinator.com/item?id=30496018 - Feb 2022 (96 comments)
Linux from Scratch - https://news.ycombinator.com/item?id=29949311 - Jan 2022 (9 comments)
Linux from Scratch with Training Wheels - https://news.ycombinator.com/item?id=28820602 - Oct 2021 (41 comments)
Linux from Scratch 10.0 - https://news.ycombinator.com/item?id=24350738 - Sept 2020 (49 comments)
Linux from Scratch - https://news.ycombinator.com/item?id=24238015 - Aug 2020 (86 comments)
Major Proposed Changes to Linux From Scratch - https://news.ycombinator.com/item?id=23787526 - July 2020 (93 comments)
Linux from Scratch - https://news.ycombinator.com/item?id=20168343 - June 2019 (15 comments)
Ask HN: Is the Linux From Scratch project still relevant? - https://news.ycombinator.com/item?id=20149111 - June 2019 (7 comments)
Linux from Scratch - https://news.ycombinator.com/item?id=16823110 - April 2018 (1 comment)
Linux from Scratch Version 8.2 released - https://news.ycombinator.com/item?id=16510333 - March 2018 (2 comments)
Linux from Scratch – build your own Linux distro - https://news.ycombinator.com/item?id=11829373 - June 2016 (57 comments)
Linux from Scratch - https://news.ycombinator.com/item?id=8392057 - Sept 2014 (1 comment)
Welcome to Linux From Scratch - https://news.ycombinator.com/item?id=4488162 - Sept 2012 (71 comments)
Linux From Scratch 7.1 Published - 3.2.6 Kernel + GCC 4.6.2 - https://news.ycombinator.com/item?id=3677350 - March 2012 (13 comments)
Linux From Scratch 7 Released - https://news.ycombinator.com/item?id=3171448 - Oct 2011 (27 comments)
Ask HN: Linux from Scratch.. Should I try it? - https://news.ycombinator.com/item?id=1779665 - Oct 2010 (58 comments)
How to build custom Linux from source code - https://news.ycombinator.com/item?id=743843 - Aug 2009 (1 comment)
I too did this, around 2003/4
It was during my great "Try all the Linux" period, and I had trouble with it compiling on my slackware system, but it compiled just fine on my red hat system (before RHEL)
It was a toy for me, I built it just to see if I could, built it a few times, but was running red hat or slackware as my daily driver.
During that period I also tried the BSDs, Free, Dragonfly, Net, and Open
It was so much fun getting the hang of how each OS differed, the nuances, the ins and outs.
(FTR, I switched to Ubuntu late 2005, and haven't moved since - apt is/was the best thing since recycled electrons)
So many good memories I had running through this way back when and it gave such a good and deep look into how a fully functioning system worked. It removed a lot of the mystery of this distro vs that distro and how all the pieces fit together. I still use some of the knowledge I gained from this in my day-to-day work, some of which is sorely lacking by others doing seemingly the same job as me.
I remember going through this and there was a point where you were running a stock, generic kernel before having built a specialized kernel with modules and options you wanted. I apparently ran up against thermal limits on this laptop because power management was one of the options for you to configure. I had to zoom through that section with a box fan pointed at that laptop so I could get power management and throttling to work so it wouldn't randomly shut down. Good times.
I used the same laptop I went through the first time with the same LFS install for a number of years after that until my day job killed my interest in doing this stuff in my free time. I switched to Debian after that and never looked back.
Like others are saying, I always recommend going through this for those that want a deeper understanding of linux, the kernel, and all its accoutrements.
for as long as i've been running and tinkering with linux, i really need to run this marathon before i get much older. i don't think it'll happen til i quit my job and have more time to actually enjoy using my computer :(
If you lack time, just go to the first part where you have an actual cross compiler. Back that up, re-use it later when you have more time.
If you have scripts that help then it is very simple. And it can be done on a weekend or two.
thanks, i'll take that into consideration when i finally get to it :)
[dead]
Making bespoke linux distros should be quite a good fit for LLM agents, especially given the recent results with the LLM-authored web browser.
Why? If you really care that little about the properties of the Linux distribution, just run one of the many that already exist.
Linux From Scratch was never really about running the system anyway -- Most people go through it as a learning exercise and then run a maintained distribution anyway; I would think it's a tiny minority that maintains an LFS system for a long time.
> Linux From Scratch was never really about running the system anyway
It definitely is. See the bootscripts:
https://www.linuxfromscratch.org/lfs/view/development/chapte...
Admittedly the main problems I had was with configuring the linux kernel. I have no good solution to make this simpler. That config file is sooooooooooo huge ... no clue how to handle this. There is no way I have enough time to sift through all the options. Or compare what changed from kernel to kernel version. Anyone has an idea how to handle this on your own?
starting from a 'known good' config file (olddefconfig, allnoconfig) and then carefully iteratively switching options around is the only methodology ive had any success with.
I've had some good experiences with using Gemini to help me explore Kernel config options (e.g. "whats the minimum set of config options i need to have enabled to faciliate feature X?")
LLMs in general are very knowledgeable about the Linux kernel (unsurprisingly!) which is why I made my original comment. You can ask them about relevant places in the kernel source tree to look at for a given mechanism, and they'll point you to the file and function without having to 'look'.
LFS is a teaching/learning tool. Asking an LLM to generate one for you would be fairly pointless. Just read the book and follow along...
Creating is easy, maintaining is difficult.
Agreed. Even with scripts, I have no idea how to maintain different linux kernel versions that are newly released.
I'm focusing only on graphics, graphics buffering, and EFI booting (and NVME if needed).