Killing the ISP Appliance: An eBPF/XDP Approach to Distributed BNG

Posted by chaz6 1 day ago

Counter87Comment24OpenOriginal

Comments

Comment by genpfault 1 day ago

Because TFA never bothered to define it:

Broadband Network Gateway (BNG)[1]

[1]: https://github.com/codelaboratoryltd/bng#bng-broadband-netwo...

Comment by bigwheels 23 hours ago

Thanks! "OLT" was also new to me. In case others find it helpful:

> OLT = Optical Line Terminal.

> In ISP fiber (typically GPON/EPON) infrastructure, it’s the provider-side device at the central office/headend that terminates and controls the passive optical network: it connects upstream into the ISP’s aggregation/core network and downstream via fiber (through splitters) to many customers’ ONTs/ONUs, handling PON line control, provisioning, QoS, and traffic aggregation.

Comment by joshbaptiste 23 hours ago

Thanks.. was reading the article like WTF is "BNG"

Comment by direwolf20 1 day ago

Is it the FTTX equivalent of a BRAS?

Comment by chaz6 21 hours ago

Yes, exactly. BRAS is functionally the same as BNG.

Comment by chasd00 17 hours ago

So what is BRAS?

Comment by lormayna 22 hours ago

I have been worked for a regional ISP 10 years ago and having an architecture like that one, would be a godsend. With centralized BNGs we were not able to apply upstream QoS policies for subscribers on the backhaulings and we had to apply policies on DSLAM access ports.

We ended using a couple of cheap Mikrotik as PPPoE concentrators for every access room, in a similar way as you did. But the reliability of Mikrotik routers was not the best

Comment by westurner 16 hours ago

OpenWISP: https://news.ycombinator.com/item?id=42953414

From "Open Hardware Ethernet Switch project, part 1" https://news.ycombinator.com/item?id=43969857 :

> There are 48+2 port switches with OpenWRT support

Are there 48 port switches with 8 or more cores?

Edit:

From "Show HN: Spliff – Correlating XDP and TLS via eBPF (Building a Linux EDR)" (2026) https://news.ycombinator.com/item?id=46663446 :

> the current "Golden Thread" correlation architecture fundamentally requires userspace + kernel cooperation that can't be fully offloaded.

Comment by dontdoxxme 20 hours ago

The code is mostly vibe coded and under the BSL. I think the interesting bit here is a single developer can write something like this with an agent. Does it make sense to open source such a thing or just each ISP write their own to their requirements?

I also don’t get the focus on handling DHCP renewals in the kernel fast path. With 2000 subscribers per OLT and say a 5 minute lease time that’s only a few renewals a second.

Comment by esseph 18 hours ago

You might have many OLTs per rack and many racks per site.

Comment by pharos92 10 hours ago

There's a commercial product available from 6WIND that makes this much more supportable for mission-critical networks. It leverages DPDK and delivers excellent performance at scale.

https://www.6wind.com/vrouter-vsr-solutions/virtual-broadban...

Comment by binome 22 hours ago

I'm curious as to what actually is the CPU <-> NPU bandwidth in these whitebox OLTs? Traditionally that has been sized for small amounts of punted control plane packets, then programming a fast path into the NPU for revenue traffic.

Comment by kjellsbells 17 hours ago

This is very elegant, but is treading some ground that for various reasons never got commercial traction.

- Cisco tried distributed BNG about ten years ago, their "cnBNG" running on their x86 UCS server line. See [0]

- A UK company called Metaswitch tried doing this with eBPF and some home-grown tech (VPP meets fd.io and special sauce) in about 2018. Interestingly they pivoted the tech to work on 5G where blazing fast user plane is essential [1]. They got bought by Microsoft, ground into glass, and wiped out five years later.

- There was a lot of talk in ~2020 about whether wireline (fiber) and wireless (5G cellular) infrastructures could converge, with the BNG becoming another node in the system, like an AGF, and authenticating users against a UDR. 5G was already very distributed and it made a lot of sense at least on market-techture slide decks.[2]

Looking back, the difficulty making this commercial was not splitting up the function, making it performant, or running it disagg on commodity hardware. The difficulty was finding a set of anchor customers who were experiencing such pain on their existing BNG that they would be prepared to jump ship from their big iron to something new knowing full well that the new system would only support 10% of what their old Lucent 7750s or Ericsson boxes could do.

Taking disagg as an example, it makes little sense unless your network is above a certain size. But if you run a big network, like DT or AT&T, say, then you will demand hundreds of features be present before you will look at an alternative. Does it work with my OSS. Does it support all the features of RFC XYZ and the special tweaks that only we have. Will it keep the three-letter agencies happy when they serve a warrant. Can it pass muster with my security people. Can the developer survive working with my procurement people long enough to make enough money to fund development.

No disruptive vendor --none-- has ever made it past this barrier into the network core, despite operators saying for years that they want to work with disruptors. That's why Nokia, Ericsson, and Huawei reign supreme and telcos haven't innovated in decades.

[0] https://www.cisco.com/c/en/us/td/docs/routers/cnBNG/cnBNG-CP...

[1] https://www.linkedin.com/pulse/why-new-approach-pure-softwar...

[2] https://www.broadband-forum.org/pdfs/tr-470-2-0-0.pdf

Comment by Nextgrid 14 hours ago

In ossified companies like telcos there's also the issue that the limitations of the existing equipment are being worked around with people. Those people derive their salaries from it, their manager derives his salary + prestige from managing such a headcount, and so on.

While the top brass might indeed be interested and benefit from more automation and a network that mostly runs itself, it's a bad deal for effectively everyone else in the company, so any attempts in that direction will never end up anywhere.

That's why legacy companies have been talking about "digital transformation" for decades now, yet it never progresses past simply digitizing the paperwork (and often creating more of it due to reduced friction), because enough people derive their job from said paperwork to make actual digital transformation politically untenable and impossible to deliver due to constant sabotage.

Comment by tucnak 11 hours ago

I mean, you see this with MikroTik all the time. The recent L3HW-enabled devices (up to 400G now) are so good it's crazy, and European onshore manufacturing, too. However, it doesn't support a subset of legacy "Enterprise" features, even though there's always a way to do the same thing using different architecture to how ISP guys have been trained many years ago, so instead we hear all the time that it's inadequate.

5G is a breath of fresh air in the sense that a lot of new techniques and broadly-applicable architectures were introduced to ISP's. I'm telling you, they HATED it. They absolutely hate learning new thing and that may as well be the largest blocker for disruptive players in the market.

Comment by heraldgeezer 21 minutes ago

I love this you and the other guy conspiracy lol. Telco bas, Ericsson bad. Okay if your stuff is so good why is it not dominant? Ah yes its all a conspiracy.

"Hate learning new stuff" = This ISP, LTE, NR stuff is all fairly new lol

Comment by WhyNotHugo 20 hours ago

Sounds like a really cool endeavour. I had no idea that ISP infrastructure was so heavily centralised. Hope the author succeeds in their quest to improve on this. I love that they're using simpler, cheaper hardware for this. Essentially, it sounds like it could reduce vendor lock-in for ISPs.

Comment by Guestmodinfo 1 day ago

Can Iran like internet ban happen? i feel the answer is no. We can finally escape govt sponsored censorship

Comment by modernpacifist 23 hours ago

The [ONT → OLT(+BNG)] → Internet] sections of the paths will continue to be owned by commercial entities that can still be the subject of court orders and/or government pressure.

Even if you were to roll your own cable in the ground to your own ONT/OLT/BNG at some point you will need to acquire IP transit or peering from other commercial entities.

Comment by direwolf20 22 hours ago

The latter usually isn't that difficult, just expensive. You can usually rent a leased line from anywhere to anywhere. The government will still come knocking if they think you're evading their censorship.

Comment by modernpacifist 21 hours ago

A leased line though will only get you A<->B where sure, A and B can be anywhere but have to be concrete locations/hand off points when provisioned. It does ultimatley come down to the service that one orders from a commercial entity.

A hypothetical court order saying something like "kill internet access" would likely cause an IP transit service to stop working (implemented by said provider no longer announcing global IP routing tables to that service) but a leased line between two locations would likely remain untouched since that isn't an "internet" service. So they might not need to come knocking if they're reasonably confident that all such edge cases like leased lines end up at dead-ends because any internet-capable product they might be enabling access to is sufficiently disabled.

I do imagine though that if they get as far as "kill the internet" that obtaining a subsequent court order to go after some suspicious leased line would be trivial.

As a side note, I find that IP transit is typically the cheapest aspect of providing an internet service since a cross-connect at a well connected DC will cost well under $1/Mbps/month unmetered. Plus the cost is very well amortized when residential users are the target. This has tended to hold when one takes into account the co-lo costs as well since network gear doing relatively basic packet forwarding/internet table routing doesn't take up that much space or power.

Comment by direwolf20 1 day ago

It can always happen. The government would just have to arrest everyone who doesn't comply, like they do in Iran.

Comment by pstuart 23 hours ago

That could never happen here /s