Framework's 10G Ethernet module exposes USB-C's complexity
jeffgeerling.com285 points by Alupis 17 hours ago
285 points by Alupis 17 hours ago
Well, this is about USB 3.2 Gen 2x2, which is a mess created by USB IF for good old, blue USB A connectors. Not USB-C complexity.
USB 3.2 Gen 2x2 is the very rarely supported 20Gb/s variant of USB 3, and making devices now that require that for full performance is a weird decision, with high-speed capable ports generally having wider support for either USB4 or Thunderbolt3+. I imagine the reason would be that some chip with an otherwise poor market fit got cheap...
Throwing this into the mix definitely doesn't improve the USB-C "what does this port support" conundrum, but this specific one predates USB-C and is not at all something you'd normally hit.
> Not USB-C complexity
3.2 Gen 2x2 (and the occasionally relevant 1x2 if you have a weak cable) are USB C only.
USB C ports and cables have 4 USB 3 "superspeed" lanes rather than two. When you use an A to C cable only one pair of those connects. The point of the "x2" modes is that they use the second pair of lanes that would otherwise go unused.
Except of course they don't always go unused. DisplayPort Alternate Mode sends DisplayPort over those two "unused" lanes getting you USB 3 data alongside a half speed DisplayPort connection. (or alternatively full speed DisplayPort on all four and only USB 2), and then of course Thunderbolt 3 and modern USB4/TBT4 use all four lanes and tunnel everything.
10 Gb/s Ethernet interfaces do not require 20 Gb/s USB ports for reaching maximum performance, they already reach that on 10 Gb/s USB ports, despite of what the writer of TFA believes.
The main application of 20 Gb/s USB ports is to connect external NVMe SSDs, when faster USB 4 or Thunderbolt ports and SSDs are not available.
For an external NVMe SSD on USB, a 20 Gb/s USB port will double the throughput, unlike for a 10 Gb/s Ethernet interface where any improvements are completely negligible.
I do not think that 20 Gb/s USB Type C ports are "very rarely supported". Every mini-PC or desktop motherboard that I have bought during the last 10 years had at least one such USB port.
Such ports appear to be rare only on laptops, because most laptops have very few USB ports.
> 10 Gb/s Ethernet interfaces do not require 20 Gb/s USB ports for reaching maximum performance, they already reach that on 10 Gb/s USB ports, despite of what the writer of TFA believes.
While this may be theoretically (almost) possible, I’m quite sure this is absolutely not the case in practice.
For example see these benchmarks of one of the more recent USB to Ethernet chipsets [1], that can reach ~9.5 Gb/s on USB 3.2 Gen 2x2 but only between ~6.2 to ~7.3 on 3.2 Gen 2x1 laptops.
1. https://www.jeffgeerling.com/blog/2026/new-10-gbe-usb-adapte...
Edit: Haha, didn’t realise TFA was by the same author as these benchmarks but he’s done a lot of testing and benchmarking of these kind of devices over a long time, and it agrees with all the other benchmarking from other people I’ve seen too!
In Ethernet, "10 Gbps" refers to the actual Ethernet frame throughput. The raw physical coding rate is usually somewhere around 10.3125 Gbps to account for this.
In USB 3.2 Gen 2x1, the actual USB packet throughput is 9.697 Gbps and the "10 Gbps" refers to the raw encoding rate.
This difference means you are guaranteed to lose at least a few hundred Mbps off maximum performance. It's not really a practical concern, but it's not an error to say 10 Gb/s USB ports lack the bandwidth needed to support the maximum performance of a 10 Gbps USB Ethernet adapter.
> 10 Gb/s Ethernet interfaces do not require 20 Gb/s USB ports for reaching maximum performance, they already reach that on 10 Gb/s USB ports, despite of what the writer of TFA believes.
The first half is true, the second half is not. Remember overhead. You don't need 20GB/sec, but you need to take into account the USB overhead.
Ethernet is duplex though. 20Gb/s is the max throughput a 10Gb NIC can achieve.
So is usb superspeed. The tx and rx don't flip around like low/full/high speed
>Every mini-PC or desktop motherboard that I have bought during the last 10 years had at least one such USB port.
Are you talking about USB 3.2 Gen 2x2 though? Because I've never seen any MiniPC with this port and as for motherboards, I checked my local retailer and only ~15% of currently sold ones have Gen 2x2 (mostly high-end ones).
Most of my mini-PCs have been Intel NUCs (or more recently an ASUS NUC). I also had some Gigabyte and Zotac mini-PCs and a few others from less well-known vendors. IIRC almost all had one such 20 Gb/s USB Type C port, unless they had one or two faster Thunderbolt ports.
With mini-PCs, I frequently use external SSDs, so I certainly used those ports at their full speed.
The only mini-PCs that I had in recent years without such a fast USB port were Arm-CPU based, as those are typically starved in fast peripheral interfaces in comparison with the Intel/AMD CPUs.
If you read carefully (emphasis mine):
> The main problem is USB-C's bandwidth complexity - especially when paired with the Realtek RTL8159 Ethernet controller, which requires USB 3.2 Gen 2x2 (20 Gbps) to get the full rated 10 Gbps speeds
Jeff's statement wasn't that 10 Gb/s Ethernet requires 2x2. It's that that requirement comes from a very specific controller.
What about overhead? Can you truly do 10Gb/s networking on a 10Gb/s USB port? Would having such NIC on a 20Gb/s USB port not result in higher speeds?
Both 10 Gb/s Ethernet and 10 Gb/s USB have bit data rates that are 3% lower than 10 Gb/s, due to encoding (64/66 bits for Ethernet, 128/132 bits for USB).
So the their maximum speed is approximately 9.7 Gb/s.
Then for Ethernet there is a protocol-dependent overhead, e.c. depending on whether TCP or UDP is used, and depending on whether standard packets or jumbo packets are used.
The TCP overhead can reach in the worst case up to close to another 3%, reducing the achievable TCP throughput to around 9.4 Gb/s.
The USB frames add some extra overhead, but it is normally not important in comparison with other factors that can reduce the throughput.
All that a 20 Gb/s USB port can do is to reduce the overhead of the USB frames, but that is a negligible improvement. Using jumbo Ethernet frames (which are 6 times bigger than standard frames), if both ends support them, is likely more useful for increasing the throughput, than using a 20 Gb/s USB port.
10 Gig ethernet is 10GBps usable rate (before packet overhead). The line rates are higher to accommodate this. For 10GBase-R, it's typically 10.3125 GBps, with a 64/66 encoding. For 10GBase-T, it's 4 lanes with PAM-16 at 800 MBaud -> 12.8 Gbps raw.
It uses 128b/132b encoding so 10Gb/s USB ≈ 9.69Gb/s you do then have USB framing overhead but it's probably around 2% on typical 1500B ethernet frames. So all in you are losing probably 5% or so to overhead.
I am of the opinion that 5Gbe is a much more sensible speed for a laptop adapter right now as it uses half the power and can obviously run full wack on 10Gb/s USB so you're looking at like 5Gbe vs ~9.4Gbe.
Stop insisting on Cat.6A (and related) copper cables for speeds beyond 1000BASE-T (maybe beyond 2.5G by now), just use dumb multi mode fiber it's way easier technology-wise and if you want power you can have that as well.
At distances where Cat.6A is even an option the demands on the fiber are very low. And it uses less power than the BASE-T PHY. The cable at least without integrated power is very thin as well, unless you can't respect it enough to not kink it, in which case you'd want a thicker one just to prevent you from being able to break the fiber.
Point of clarification since it isn’t clear from the title. This isn’t a Framework product, but a product by Wisdpi designed for the Framework Expansion Card form factor.
Very cool to see an ecosystem developing around that form factor. Like the old PCMCIA, except not awful.
I do wish there were something like Oculink, but with power available over the connector. USB-C does almost everything, but it seems the chips to break out PCIe lanes for USB4/Thunderbolt for higher speed devices are still a significant cost for accessories.
hey Nirav, dumb question: would it be possible to have usb-c ethernet adapters using intel chips in order to have vPro features on framework laptops (along with vpro-enabled intel chips) ?
That's probably the missing cherry on top, as having vpro once the framework motherboard gets reused as a home server it gives some manageability features.
vPro requires the NIC to be connected directly to the PCH (over PCIe/CNV). This is going over USB, which won't cut it.
An Intel WLAN card in an M.2 slot on the mainboard might work (given your givens; a vPro enabled chipset).
I chuckled at 10G wired ethernet on a laptop. I mean in a docking station? Sure that seems reasonable. But fun none the less.
I appreciate the USB-C nature of the Framework's expansion ports, it does make real the entire reason that USB was created in the first place, hot plug slots. Still, I (and others) pointed out to Intel early on that using Ethernet with a specific packet type would be cheaper and just as fast (which the ATA over Ethernet folks proved), but then you wouldn't get the 'certification tax' that the USB consortium extracts. :-).
Cynicism aside, the design issues suggest that it might make sense in future laptops to have heat spreaders around the plug in port, although that makes things thicker and people obsess over thinness.
The problem with docking stations is they're more expensive than an ethernet adapter. I tried to use a few 2.5+Gbps dongles with my laptop(s) to avoid spending $400 on another Caldigit dock (TS4; I already have a TS3 with 1Gbps ethernet).
Unfortunately, all 3 USB-C dongles I tried had significantly worse performance than the built-in 1 gig ethernet on the dock, apparently using the RTL8156 chipset which is known to be unstable.
I've got a 4th dongle on the way to try next! If I buy enough of these things I'll have spent more than just buying the right dock in the first place.
> I chuckled at 10G wired ethernet on a laptop. I mean in a docking station? Sure that seems reasonable. But fun none the less.
What difference does a docking station make? Sometimes you want to spend a minute or two setting up your laptop in a more serious way, and that's just as reasonable with or without a docking station.
The "dock" comment made sense to me because I don't think that true "road warrior" laptop use and 10G Ethernet deployments would coincide all that often.
I've put a disproportionate number of hours and $$$ into my homelab over the years, and I still only have 2.5G Ethernet switches deployed. Most offices' (much less home/coworking space/etc.) network traffic is passing through single-gigabit switches.
I have a 5gbps symmetrical fiber connection at home, so I've spent a fair amount of time and money upgrading my homelab backbone to 10gbps. That includes a 10GBe connection to my desk, but I've had issues getting the connection to be reliable (terminating the shielded Cat6A I have in the walls is a pain). That drop hasn't been working for the past few months, so I've been on wifi instead; it hasn't been enough of an issue for me to invest the time in fixing it.
>I chuckled at 10G wired ethernet on a laptop.
Back in the early days of wireless networking I had my laptop configured with the wireless and wired networks bonded. I want to say that was 2Mbps on the wireless, so if I was doing a big transfer I could walk over to a wired port in my house and plug in to get 100Mbps.
Others comments already mentioned multimedia, but for example where I work we have some development board and prototypes with 10g ethernet, but most developers have a laptop rather than a fixed station. Turns out smallish (but overly expensive) thunderbolt 10g adapters can be used for testing and even reach full thoughput in many cases.
If you work with media having a 10G connection on a laptop isn't all that absurd. In fact slow network speeds are the main reason why people have to use things like Thunderbolt instead of using a NAS (e.g. offloading data on a film set).
My laptop is basically a desktop in a clamshell so I don't need a docking station (and it's like 5 lbs with crazy cooling so I can use it as, you know, a laptop). I work with large binaries (media) on a daily basis. I have a 1G ethernet port built in, but I'd love a 10G port. I'd absolutely make use of it (maybe not all 10Gs of it, but most of them). Besides, things are only going to get faster...
I have a Thunderbolt 10Gbe adapter. It's a larger form factor than a Framework expansion card and it has a metal case, so it dissipates heat well. Copper 10Gbe chipsets generate a lot of heat.
Ethernet comes with a completely different set of social norms, like not having a master and slave device.
I wish they would just have direct PCIe lanes. For most of the cards doing those speeds USB is just unnecessary overhead
They sort of do in the 16 with the dual M.2 adapter:
https://frame.work/pl/en/products/dual-m-2-adapter
People have been making custom OCuLink adapters and recently Framework developed its own:
Future proofing. Websites nowadays load tens of megabytes for a simple news page. I wouldn’t be surprised if in the future a website consumes 500+ megabytes on the initial page load. That will take 40 seconds on a 100 megabit link and 400 milliseconds on a 10 gigabit link.
Try to load any modern website on dial-up. The connection will likely timeout before a full page load.
This is the most dystopic use for that bandwidth I could possibly imagine.
Fr. There was that small window back in the early ‘10s where data was slow and expensive in India and 4g wasn’t yet pervasive in China. Execs were still receptive to my pitches that we needed to spend some engineering cycles on rework to improve initial load times / weight after showing them simulations of absurdly slow first paint / first content over simulated 2g/3g from India / China data centers.
Once 5g became pervasive and data cheap, no one gaf about a cold load weight fitting on a floppy.
I’m still clutching my iphone mini, which after ios 26 just boggs down under the absurd weight of many pages and turns in to a space heater before reloading entire page b/c of error. No need for forced obsolescence when the enshittification of basic websites takes care of that for you. :-/
Step 1 of displaying a web page is to download a small LLM from the server to create a rendering framework at run-time...
Every PCIe 10G ethernet card I've seen has a heatsink on it, sometimes covering the entire card or even have little fans on the heatsink.
Expecting it to work full time in a laptop is a bit of a stretch of the heat dissipation budget.
Also, the laptop he is working has the AMD FP8 chipset - depending on how the ports are setup, he might only get 10G USB, if the ports are allocated to video instead.
New chips from Realtek burn < 2W for the chip and < 3-4W for the board: https://www.servethehome.com/cheap-10gbe-realtek-rtl8127-nic...
4W is TDP for some of Pi-style mini computers. Lots of them have fans.
Pi 4 and 5 both idle around 3W. But a Pi 5 can pull up to 16W with a USB peripheral, full CPU load, and decoding 4k video. The Pi 4 / 5 will run OKish without a heatsink at idle wattages, but thermal throttle quickly if you attempt to do something intensive.
These realtek 10gbe chips are more in the range of the Pi Zero class machines (0.5W idle, 2W loaded) which don't often come with heatsinks though they might benefit from them. If it has a good thermal connection to a good thick ground plane on the PCB, that's worth almost as much as a passive heatsink on the top of the chip.
usb-c < card edge < motherboard integrated in terms of how much heat can be transfered through the connection. Where the motherboard would have the largest ground plane to soak up heat from such an IC and dissipate it passively. The usb-c module is worst case by being a small enclosed box with very little thermal connection through the plastic insulating housing. An aluminum enclosure might dissipate enough heat passively to make it pleasant to use.
> The Pi 4 / 5 will run OKish without a heatsink at idle wattages, but thermal throttle quickly if you attempt to do something intensive.
Even with a heatsink and fan, I had to upgrade to a higher quality set to keep Jellyfin from thermal throttling a Pi5 while transcoding 4K video.
4k video transcoding is anything but an idle load.
Especially on the Pi 5, which has no hardware encoder to save on power consumption for that task. It's entirely in the CPU.
(Technically the Pi 4's hw encoder doesn't go up to 4K either, though, so I guess moot point).
Raspberry Pi 4 doesn't need a fan. People just like to put them on because because micromanaging CPU temperature is part of the hobby for some. Yes it might throttle its CPU speed when going full tilt for some time, but lets be real how many workloads require poor Raspberry Pi to be loaded 100% for prolonged periods of time?
If it throttles CPU it means by definition means that a fan helps. Also constant heat increases failure rate.
Cycles of heating and cooling are what increases failure rates. The thermal expansion and contraction causes issues.
That's one way to put it.
Another way is that my great grandchildren won't care about inheriting my collection of hobbyist SBCs, and therefore nor should I.
Permanent heat doesn't?
From what I've gathered, heat absolutely does[1] affect[2] it[3]:
Subsequently, in 1967, Black of Motorola experimentally derived a median time to failure (MTTF, i.e., operational lifetime) model for EM in Al interconnects, showing that the time to failure due to EM is inversely proportional to both the current density and the absolute temperature of the interconnect.
[1]: https://infinitalab.com/blog/ic-failure-analysis-defect-type...
[2]: https://resources.system-analysis.cadence.com/blog/msa2020-b...
[3]: https://www.mdpi.com/2079-9292/14/15/3151#sec3-electronics-1...
Thermal cycles, heat, current, all contribute to degradations and failures. It just so happens that cycling is the worst and everyone knows "it's the power cycles that kills computers". Doesn't mean at all that electronics can't be damaged countless other ways.
Does it go beyond 30 Metres?
Certified Cat6 cable gets you 10GbE up to 55m (and even 5e is workable), while Cat 6A goes to 100m.
* https://en.wikipedia.org/wiki/Ethernet_over_twisted_pair#Var...
...and yet they're still covered by a huge heatsink.
To add perspective, an old-school 7805 voltage regulator dissipating just 1 watt is already impossibly hot to hold with bare hand (as me how I know). So 3-4 watts on a small module will make it noticeably hot.
They aren't huge at all, the new RTL cards are tiny. I wish 2-port versions were available for a home server upgrade.
I was doing a comparison of 10G ethernet NICs just yesterday and ChatGPT was insistent that they are scorching regardless of actual throughput. Unless you manually downshift and upshift the communication rate.
I'm having second thoughts about having one of those dongles on my desk all day for the same reason wireless charging seems wasteful.
Yeah, 10Gb ethernet runs hot. I just rewired the house with 10Gb (we have 8Gb FTTP) and it's kind of upsetting how hot my Thunderbolt dock gets.
I looked in to it and it seemed like 10gbit was much better over fiber. Ended up deciding that 2.5gbit is plenty. The 2.5 gear is significantly cheaper and runs cool.
Yeah, I use DAC for the desktop and fibre between floors. It's just the Mac's desktop that uses RJ45 copper.
> I looked in to it and it seemed like 10gbit was much better over fiber.
Yes, except that most devices use Ethernet. So, at the end of the day, you still need Ethernet cables unless you want to deal with an additional switch or converter in every room.
Fiber/10Gbase-*R is Ethernet too. Please say copper/RJ45/base-T when you mean copper/RJ45/base-T.
If you want to ackshually, both fiber and copper are empty pipes that can carry any layer 2 protocol, and are not inherently Ethernet. They only become Ethernet cables when they're connected to terminals that pass that protocol through them.
Unless we're defining some networking standard, "Ethernet cable" is a perfectly acceptable term. Everyone will understand what is meant. The added specificity you're asking for doesn't improve the quality of communication.
That's why I added base-*R/base-T.
And particularly for 10GE the heat and power problems are due to the copper transceiver DSPs.
And people nerdy enough to run 10GE at home might well run fibre.
So, no, the specifity is needed and useful.
>That's why I added base-*R/base-T.
You're still talking about a cable. The cable may be compatible with those standards, but you can put anything through it. It's just a physical connection.
>And people nerdy enough to run 10GE at home might well run fibre. So, no, the specifity is needed and useful.
No, because if you say "which do you want? Ethernet or fiber?" no one will look at you like if you asked if they want salt or beef. It's technically incorrect, but everyone will understand what is being asked.
> You're still talking about a cable. The cable may be compatible with those standards, but you can put anything through it. It's just a physical connection.
If you want to ackshually, the post I was replying to was talking about what "devices use" and cables required for that, so it's in fact about what standards these devices support.
Apart from that, again, in the context of 10GE you can by no means assume copper when talking about an Ethernet port; SFP+ slots are quite common. Your assertion that "everyone will understand" is also something I plainly know to be untrue in my bubble. It may be true in the context of slower speeds, but for ≥10GE the general performance characteristics of twisted-pair copper transceivers are so bad as to make it into the crossover point from copper cabling into DAC cables and fibre.
And, honestly, the assumption that "Ethernet = copper cabling" is harmful for 10GE. Those transceivers are hot garbage in the literal sense, they run hot enough to warrant usage limitations on switches due to cooling/overheating limits, and they tend to be quite picky about cable quality on establishing links.