Getting 25 Gbps Thunderbolt Ethernet on My Mac Studio

I initially read this as SONET[1] and was seriously questioning your technical choices.

[1] https://en.wikipedia.org/wiki/Synchronous_optical_networking

ConnectX-4 and newer are natively supported. I use a ConnectX-4 LX in a $50 Razer CoreX off of craigslist. Works fine, hit similar limits to what Jeff did.

Using an OCP is the neat trick here. MUCH smaller setup.

As far as the other posters comment about Tx's... that is an entirely different topic. I've never had an issue with FS.com optics personally.

If you are unfamiliar, the reality of the situation is quite bad. Slight hardware revisions of adapters/switches, which if you are lucky can be differentiated by manufacture date or serial number sequence, will not recognize known good modules, and if they do you then have to actually test if they can establish link, then let them burn for 24+ hours to see if they can maintain it without dropping (often, because of forward error correction, you will not start to see flapping until enough correctable errors have accumulated and the buffer overflows). Then there are firmware updates where this process must then be repeated.

You would think with how expensive all of this shit is you would simply be able to buy the officially listed as compatible components but my experience with them has been that a shotgun approach is best for your sanity and deadlines, buying sample kits of each module and trying them all, and then once you find the ones that play nice, buying as many of them as is reasonable and testing all of them.

If I had a dollar every time two modules from the same brand, that came in the same box, from the same production run, did not work with one another, I would be able to test your theory for free. Network Engineering is Fun

His iperf3 results show that it's able to go full beans on a single 25g port. A cross platform SMB file copy is kind of a meaningless benchmark in this context.

IMO a pretty cool product would be a network switch providing native thunderbolt interfaces to multiple upstream hosts. That wouldn't magically fix this RDMA issue, though. This is just a separate thought. Seems like the kind of thing Mikrotik should make.

I found that SMB on macOS is just bad. If I try to heavily load it (e.g. by doing mass conversions in my Audible library), it can cause stalls and disconnections.

I managed to find a set of parameters (in /etc/nsmbd.conf) that makes it work more-or-less reliably by trial-and-error: dir_cache_off=yes, file_ids_off=yes, notify_off=yes, mc_on=no.

It's still terrible, though. It's so bad that Linux in a Parallels VM works _better_.

MacOS has supported RDMA over thunderbolt since March 19th.

See also NVMe drives? They draw enough idle power to drain a laptop battery overnight, and will easily hit 60+C in a bad enclosure. That’s only a 10Gbps module. The USB4 enclosures are all heatsink.

And your computer already speaks the PCIe and thunderbolt. It has that probably on CPU. Here we have to receive USB, convert it to Ethernet. On much less up to date process nodes than your CPU.

Third, until very recently for even 10Gbit it meant using a usb4<->pcie bridge chip which itself was hot, to provide a generic pcie connection, which then talked to a conventional network chip, so it was a double hop: USB <-> pcie <-> nic. I don't know if 25Gbit to USB direct convert is available yet but we've seen a wave of vastly cheaper 10Gbit adapters come out that are single chip USB<->nic, probably not pcie at all, that are much much cooler. 25Gbit is probably still not here yet, maybe?

With ethernet it's for lots of reasons but a big one is not creating ground loops between distant equipment. If you didn't isolate the network cable would be sinking potentially amps of current between distant devices and that just isn't sustainable. You don't have this problem attaching USB between two devices.

The NIC in this setup was released 12 years ago. It's built on a very old process.

you can look up the rated power use of a connectx4 card, they do a lot of offloaded computation, too

I believe MacOS supports it via the usual network control panel. I’ve used it for transferring data from an older laptop to a newer one. The cable is more expensive than fiber, last time I checked.

Latency was an order of magnitude higher than twisted pair but the throughput was greater than 10Gbps and I didn’t have to spend on 10GbE capable servers and network equipment. I vaguely remember that they had to be proper Thunderbolt not just USB-C but that could have changed it’s been quite a while.

[1]: https://en.wikipedia.org/wiki/RNDIS

But for all certified Thunderbolt 4 and 5 ports, this kind of networking support is a requirement. (It is optional with Thunderbolt 3.)

The rest is up to the operating system.

On a CalDigit TS3 dock, I've had one working off the rear Thunderbolt socket for over a year, but it fails in minutes if plugged into the USB-C one. I've never had issues with a CalDigit Elements (TB4) hub. I've had success plugging them directly into various Chromebooks. On one Windows laptop, they seemed to work on one USB-C port but not the other.

So... reliable once working, but getting there can be a pain.

For my Macs, the IOCrest USB4 adapters with the AQC113 chipset have been rock-solid and can saturate my 10Gbe network. They do run hot but no stability issues. That's what I use and recommend.

The newer RTL8159 based dongles are smaller, cheaper, cooler, and can do 10 Gbe on paper. But they top out at 7 Gbps on Macs, which don't implement the USBx2 spec.

Buying a Mac with built in 10gbe is so nice compared to dongle hell.

But it’s not 25gbe

LLMs: https://www.jeffgeerling.com/tags/llm/

Etc... (I don't do any crypto)

Edit: https://www.jeffgeerling.com/blog/2025/15-tb-vram-on-mac-stu...

Obviously it's possible to move this to a NAS, not use a mac, etc. etc. - but just wanted to chime in with my little use case of moving multi-gig-files around. :-)

I was just looking for a thunderbolt-sfp solution. Super timely!

That'd be the rate for 1 stream at 1x playback. Multiple streams because of overlays/crossfades/etc... along with playback at >1x (such as during rendering in particular) will change that significantly.

A typical NVMe SSD can easily do sequential reads at 7GB/s, which outpaces a 25Gbps connection by more than 2x. NAS is more typically backed by SATA HDDs, but there's some relatively easy way to put an NVMe disk in front of them as a cache.

Do you ever _need_ it? Perhaps not, but there's plenty of common activities just stemming from NAS usage which see a linear speedup, and would continue to get faster up to 100Gbps even.