Running Your Own As: BGP on FreeBSD with FRR, GRE Tunnels, and Policy Routing
blog.hofstede.it113 points by todsacerdoti 8 hours ago
113 points by todsacerdoti 8 hours ago
Not to nitpick, but the title should have AS capitalized. It’s confusing with the current capitalization.
Right! I was confused for a bit until I started reading it.
Otherwise, getting to know the power of FreeBSD is awesome. Thanks for creating the blog!
I think HN tends to undo all caps words unless it's an acronym HN specifically recognizes. Guessing BGP, GRE, and FreeBSD are understood but AS is not.
It’s too late now, but when submitting a post the poster has a window of time to edit the title. Useful for example when HN auto-edits to capitalisation get some words wrong. When you edit the title, those auto-edits are not applied to your edited title.
If you'd like to experiment with running your own AS in private address space, connecting to a friendly network of geeks over wireguard tunnels, check out DN42 https://dn42.dev/Home.
It's a great way to explore routing technologies and safely experiment with your own AS, running the same protocols as the "real" Internet, just in private space.
If you do get set up, give me a shout (https://markround.com/dn42), I'd be happy to peer with you if you want to expand beyond the big "autopeer" networks :)
I was hoping with IPv6, getting an address space as an individual would go back to how it was in the early IPv4 days, but alas you need to be a multihomed individual with tons of usage instead of just a sophisticated netzien that wants to own their block.
I don't want an address, they should be cheap, meaningless (sans routing, the longer the common prefix, the closer geographically you should be) and not conflated with identifiers.
I just want a way to do public-key based discovery. I'm not sure if wireguard + DHT would do though as it'd also mean that it's easy to track your PK (and maybe you through your devices/services announced with PKs).
Maybe you can announce your IP in a neat encryption scheme that adds some privacy without increasing costs too much?
Yes, same here. Very frustrating. It is almost as if the powers that be don't want lowly netizens controlling their own destiny.
Actually, they don't want to pollute the internet routing table with routes that are fully subsumed into other routes. The effect on address ownership is a side effect.
Actually, they just want to milk the money out of you. It's a matter of how much your willing to pay, as a business customer, it's all possible.
Most ISP do not have such pure goals, as to protect the global routing tables ;)
RIRs, not ISPs, allocate addresses at the top level, they make money on each address allocation, and they still won't allocate addresses to you if you don't multihome because they have a duty to conserve resources.
When you get PI addresses your LIR/ISP just passes your data on to the RIR.
What is the point of owning public address space?
Anything in your private network (even if it goes over public internet) should be encrypted and locked up anyway. Something like Wireguard or Nebula only needs a few (maybe just one) publicly accessible address. Inside the overlay network, it's easy to keep IP addresses stable.
Anything public-facing likely needs a DNS record, updatable quickly when the IP of a publicly accessible interface changes (infrequently).
What am I missing?
The realistic point is to have your own abuse email contact, to evade the banhappy policies that most server hosts have even when you did nothing wrong. Usually they suspend your account if you don't reply within 24 hours, even if the complaint is obvious nonsense.
It's the only real way of running reliable IPv6 networks with multiple uplinks. Unless you want NATv6.
Honestly it's not free but it's really not that expensive. With RIPE it's about 75€ per year for the ASN and being multihomed is not really a problem, there are multiple services that will let you announce through them for free or very cheap. You don't have volume minimums.
I do agree it should be simpler, but it is accessible to individuals today.
`-rxcsum -txcsum -rxcsum6 -txcsum6 -lro -tso`
Why disable all offloading? It's not explained anywhere.
Poor driver support on the poster's particular hardware, maybe?
In that case they should add a warning there in my opinion. It makes a lot of difference in my testing
I do a "light" version of this, but without running a public AS and using WireGuard for tunneling my public IPv4 subnet into my homelab (proxmox cluster).
Just running bird on my VPS to announce my routes to the upstream over a private link.
I looked into buying my own IP space from that IP auction site, an IPv4 C-class costs around $10,000. What stopped me was finding out I also to register with RIPE and pay the LIR annual fee, costing hundred Euros per month or so, even if I wasn't yet ready to use the IP space (I wanted to setup a basic Anycast IP without Cloudflare with help of VPS host who said they can help and had multiple locations around world).
While I strongly support IPv6 migration, the current IPv4 pricing is a rip-off. All the brokers and auction sites are fantasizing.
The market is tight, but nowhere near the point where it was 4-5 years ago. Big cloud providers already bought enormous amounts of IPv4 while many regional ISPs and colocation providers went out of business.
There is no real pressure to buy IPv4 except for brand-new companies to get their initial /24 or /23 to start. Everything else is optional.
How can an auction site fantasize? The price is what someone bid, and that's the real price.
When I bought my initial /24 on such a site, it was not a competitive auction. I was the only bidder, and I paid the opening bid price, which was set by the seller. It's true that it was a real price, in that I paid it, but the 'auction' aspect felt like a farce.
They keep details private. It's not something transparent like eBay or a public auction. I think it's just a scam to pressure buyers into offering more.
If you can register on ARIN the costs are only $260/year at the smallest tier and you can also apply for a /24 which you should be able to get.
You only need an LIR annual fee (~$2000) if you want to be an LIR and manage other people's resources. Otherwise you find another LIR (some popular choices are the ones the OP used) to manage your resources on your behalf. The annual fee is then ~$60. The resources are allocated directly to you, even when managed by a third party.
If you have a ham radio licence (anywhere in the world) you can request a /24 if IPv4 space from AMPR for free.
It cannot be used commercially and should be in the ‘spirit’ of amateur radio. Unfortunately there’s also a bit of a backlog it seems (a couple of months) right now.
Oh, interesting. What's at the intersection of networking and amateur radio that these address blocks are often used for?
Quite a lot of interesting stuff - for example there are mesh networks setup worldwide that attempt to run IP over RF using these - and then use the internet to forward packets from one to another.
They also offer simpler ‘turn-key’ wireguard tunnels too for things like Web SDR setups.
For BGP direct announce in practice it seems to be in the spirt of non-commercial ‘self learning and experimentation’ which is what a lot of legislatures around the world do use as their base definition for the ‘amateur’ in amateur radio. So I guess much like having slices of radio frequencies reserved for it, we’re lucky there are slices of address space reserved for this.
Note that it is not a real C-class IP prefix unless it is from the 192.0.0.0/3 range, otherwise it is just a sparkling /24 IP prefix.
Yeah for single person use, this only really makes sense with IPv6. I'm interested in doing this in the near future and I think the yearly price for all-in (IPv6 /48 allocation, AS allocation + necessary VPS connections) comes out to about $200. It goes up to $300-400 if you want a PI subnet instead of PA (PI follows you to another LIR, PA does not).
> MSS clamping is non-negotiable with tunnels. Every layer of encapsulation eats into the MTU.
Can this tunnel be avoided somehow? If I have to choose between owning my prefix and having 1500 MTU, I'd probably take the latter: MTU issues are so annoying to deal with, and MSS-clamping doesn't solve all of them.
Kind of but not really.
The whole point of BGP is to influence your routing tables. This fundamentally makes very little sense to do when you have a bunch of routers whose routing policy you don't control between you and whoever you're speaking BGP to. eBGP is just TCP and supports knobs to run over multiple hops (so up to 255), but at that point you can't really do anything with the routing information you exchange because the moment you hand the traffic off, the other party can do with it how it pleases. Also, very few people have enough public IP addresses for this, and on the Internet you obviously can't route RFC1918 space. Therefore, you need tunnels, so that you can be one hop away even if the tunneled traffic is traversing the Internet, and so that you can reach peers that let you announce whatever IP space you want.
The other thing you can do, of course, is to just do the same thing internal to your lab. You can absolutely stand up multiple ASN at home. I'd even argue that if you really want to learn BGP, this is a great way to do it, especially if you use two different platforms (say, FRR on FreeBSD peering with a cheap Mikrotik running RouterOS). That way you learn the underlying protocol and not a specific implementation, which is something that is very hard to undo in junior network engineers that have only ever been exposed to one way of doing things.
That's different from some of the goals outlined in the article, but if your goal is to learn this stuff rather than have provider-independent IP space (which even for home labs isn't very valuable to most people), doing it all yourself works fine.
You can use who you're physically connected to. If you have a physical or point–to–point connection to iFog and Lagrange Cloud, you don't need tunnels to reach them. Both these companies offer VPS services.
If your goal is to learn this stuff join dn42, the global networking lab, instead of wasting money with real allocations.
iFog and Lagrange Cloud, naturally.
I am always very curious why these operations exist. ISPs for the very specific niche of hobbyists who want to run ASNs.
Just a reminder, that the basic fees at RIPE are 2-3x the fees at ARIN which hurts individuals, SOHO and multihomed not-for-profit institutions.
fee schedules FYI
- ARIN 2026 PDF: https://www.arin.net/resources/fees/images/2026feeschedule.p...
- RIPE 2026 : https://www.ripe.net/membership/payment/
Enthusiasts, trainees and small orgs are paying a lot more with RIPE.
If you want to be an LIR and have the right to manage other people's addresses on their behalf, as well as being a full member of the organisation with voting rights and so on. If you just need addresses, that's not you.
Your ARIN link is broken.
fixed arin link: https://www.arin.net/resources/fees/fee_schedule/
It's basically $275/year to have an AS and some PA assignment with no intermediary LIR. In Europe, you have to pay €1800/year without an ASN included. Each resource is billed separately. If you go with a middleman (another LIR) you usually have to pay 200€+ (with taxes) for 2 resources (ASN and PI space)
Good to know. As someone on the ARIN side, I always found the fees reasonable.
You can get better deals with the right LIR. As a hobbyist it was cheaper for me to go with a RIPE LIR over ARIN.
It's not comparable. You will lose your AS and PA if your sourcing-LIR goes out of business or increases prices against you. It's ab big difference to become a LIR or just a downstream customer.
You shouldn't lose an ASN or PI block, they are registered to you at RIPE, only managed by the LIR and can be transferred to another LIR in exceptional or routine circumstances. I think you'll have to pay another fee though.
A PA block is just part of a LIR's block that they give you permission to use, so I doubt you could keep that if they went out of business, but maybe RIPE has a procedure for it.
For a hobbyist it’s perfectly fine, I think? I’ve been doing this for years. If I was a major corporation I might be more concerned.
[flagged]