NetBSD 11.0

I am sure there are other impressive stories about what people did with NetBSD, too bad these really never get published. Things like this would have been nice to show up here:

https://www.netbsd.org/gallery/research.html

All your Netflix video streaming from Open Connect CDN Edge are served by FreeBSD. [1] Serving at 800Gbps per Server.[2] This was 4 years ago, drewg123 also hangs around on HN so I hope he read this and there are some 1.6Tbps update soon :).

NetBSD runs on everything. So it's part and sometimes NetBSD itself is running only vanity of embedded systems. A lot of these information are pre 2020. But given how slowly the embedded industry moves I wouldn't be surprised if it is still largely the same. I do wonder if it was grown though. It is also the only Rump kernel implementation that I know of.

OpenBSD ends up a lot on HN, many part of our every day internet usage are done with components maintained by them. There are a lot of network appliance and server usage using OpenBSD.

I think BSD deserves more credit, but for its license and philosophy I guess is it mostly working as intended.

[1] https://freebsdfoundation.org/netflix-case-study/

[2] https://news.ycombinator.com/item?id=32519881

Another cool corollary to this ethos is the portability of their portage, pkgsrc. I’ve personally used it as the basis of my user space packaging system on a Linux From Scratch distribution I made during the pandemic and the results were awesome. I could be wrong but it is the only package ecosystem I’m aware of that currently supports MacOS, Linux and the BSDS.

As far as the health of the ecosystem pkgsrc uses a rolling quarterly release process, has built in mechanisms for identifying vulnerable packages and a dedicated (volunteer?) security team and packages are about as up-to-date in many cases as, say, arch or gentoo.

Additionally pkgsrc can run rootless and my understanding is it finds itself used in a lot of high performance scientific computation settings as a result.

Naturally I also use obsd as my firewall, a role I would recommend it in. at least if you don't mind a bit of old-school unix administration.

I have a freebsd fileserver and never got around to trying netbsd, But between fbsd and obsd I found I prefer obsd as the desktop system, it is a subtle thing hard to describe, but it feels better focused for desktop use than freebsd, not as performant and the filesystem sucks in comparison(thus the fileserver) but somehow feels better. I don't know.

DragonFly BSD ([1], [2]): "Am I a joke to you? :-("

;-)

By the way: If you look for a BSD-based desktop OS, have a look at GhostBSD ([3], [4]) (based on FreeBSD) and/or MidnightBSD ([5], [6]) (forked from FreeBSD). Both are under active development.

---

[1] https://www.dragonflybsd.org/

[2] https://en.wikipedia.org/wiki/DragonFly_BSD

[3] https://www.ghostbsd.org/

[4] https://en.wikipedia.org/wiki/GhostBSD

[5] https://www.midnightbsd.org/

[6] https://en.wikipedia.org/wiki/MidnightBSD

I know some people use OPNsense or pfSense for their router and firewall needs, which are both based on FreeBSD.

The setup on openbsd was very straightforward. I had to read up a bit and claude helped getting ipv6 and vlans up. Had I ignored ipv6 it would have been ready in half an hour from just reading man pages and official docs. The only reason ipv6 took longer is because I had no idea how it works.

I researched the equivalent setup on linux and it was much less straightforward. In openbsd I only used software in the base install. For linux there were a lot more choices to make, which for me just made setting it up more difficult. I don’t want to pick a firewall, I just want to configure the simplest possible one.

The only issue I had was the system hanging when no monitor was connected to the hdmi port. I use an old apollo lake mitac pd10ai itx board. Displayport worked fine during install but black screened before I got a shell and crashed. Tried setting some kernel options to disable some display driver without success. Opted to get a dummy hdmi plug instead for reboots without a monitor.

Also the subsystems are way simpler (packet filtering, sound, network management). Linux is full choke of abstraction which aim to serve a huge range of devices. And I truly believe that there should have been a bifurcation somewhere.

Most software are easy to port around, but there are some that relies on linuxisms (udev) and some on glibc-isms, which can require some patching. Some of the pains I encountered are about device files (usb) and the fact that openbsd relies on root a bit more (mounting with fuse).

You can read the “Absolute OpenBSD, 2ed” book, published 13 years ago and almost everything still holds. You can even read the “Design and Implementation of the 4.4BSD” and still have a good understanding of the OpenBSD design. Linux is chaotic land compared to that.

i.e. your PC is unsupported for some specific driver, such as bluetooth? stick AI on the problem.

FreeBSD (and all the BSDs really) do not use systemd.

> How do they compare to Linux these days

They don't use systemd.

> How do they compare to Linux these days, in terms of size, feature set, security hardening, etc?

Security harderning? They don't have systemd. So there's no backdoor in SSH because somehow sshd is linked to some compression library because a Microsoft employee decided Linux should use compressed logs.

Really: no systemd is a big one.

Until around 2004, I looked at Linux more like a toy in comparison with FreeBSD, because the latter had much better reliability and performance.

However, in the early years after 2000 Linux has evolved and it gained more and more advantages over FreeBSD, mainly caused by a much greater number of programmers that contributed to Linux.

The first that affected me personally was that gradually Linux added support for a much greater number of peripheral devices.

Unfortunately, after the launch of Windows 95 all vendors of PC peripheral devices have stopped providing technical documentation for their products, replacing that with Windows drivers (or in rare cases, e.g. for network interfaces, they might provide drivers for a few other operating systems).

This means that for non-Windows OSes someone has to reverse engineer the device drivers. In the better cases, there are some standardized peripheral interfaces, like USB, ATA, SCSI etc., but that is not good enough for using a given peripheral device, because most of these have always been full of undocumented bugs and they work on Windows only because the Windows device drivers contain workarounds for the undocumented bugs.

This means that the Linux kernel includes a huge database of quirks with the bugs of various peripheral devices that exist on the market.

Eventually the BSDs were also forced to include such bug databases, but theirs were and still are much smaller than those of Linux, because they have less users, so less people who might encounter the buggy devices, investigate the cause and report it.

For example, I had some Athlon CPUs that I used with motherboards bought in 2001, which had VIA chipsets. In 2000/2001, the VIA chipsets had a horrible bug that caused data corruption on hard disks. There was a workaround to serialize some of the ATA commands, to avoid their concurrent execution that resulted in data corruption. Of course, VIA kept secret the existence of the bug, so who knows how many people have lost precious data because of this.

When I first bought a motherboard with a variant of the buggy chipset, for a Pentium III CPU, by chance I discovered very early the data corruption on Windows, and I returned the MB. I bought another MB with the same chipset from another MB vendor (Soltek), and I blacklisted the first MB vendor (Soyo), after not seeing data corruption on the new MB.

I was very wrong because the MB vendor had no responsibility in this, only VIA was guilty. The bug disappeared on the second MB because meanwhile VIA had updated its Windows driver. So they caused lost sales to the first MB vendor, by their secrecy.

Sometime later, I decided for a MB with Athlon to be my first desktop PC that was not configured for dual-booting with Windows, so I wanted to install on it only a *BSD or Linux.

However, neither FreeBSD nor NetBSD nor OpenBSD worked on it, all had the data corruption problem. But Linux worked fine, so I installed Linux and since then I used only Linux for PCs, restricting FreeBSD for certain kinds of servers.

Only later I discovered the reason why Linux worked, unlike the BSDs, which was because it was aware of the bug existing in the VIA chips and it included the same workaround as the Windows drivers.

I believe that my experience was typical of the positive feedback loop that enhanced the market share of Linux, more users resulted in good support for more hardware devices, which enabled the adoption of Linux by more users, and so on.

Then the major blow that hit the BSDs was the launch of Pentium 4 with SMT in 2003, followed by the dual-core AMD CPUs in 2005.

On a single-thread CPU, FreeBSD 4.x had a much better performance than the contemporaneous Linux, for networking and storage operations.

But the design of the *BSD kernels was incompatible with multiple threads and the rewriting of the *BSD kernels to work with multi-core and/or multi-threaded CPUs took several years, while Linux had already been ported to SMP architectures and it took immediately advantage of the new fast CPUs.

During the first decade of this century Linux took a great leap in performance beyond the BSDs, and they never caught up completely with it after that.

Beyond the fact that it worked well with many threads, Linux introduced during those years many improvements over the traditional UNIX-derived features.

The most original was the use for synchronization of compare-and-wait-for-mutation, a.k.a. futex (2002), which was the correct solution for a problem discussed by Dijkstra 40 years earlier, but for which Dijkstra had proposed a wrong solution, i.e. semaphores, a solution that caused inefficient implementations of IPC for decades (Dijkstra had criticized the use of event flags for synchronization, because of a possible race condition, which was avoided by the commutativity of the increment/decrement operations on semaphores, but compare-and-wait avoids the race, without using superfluous auxiliary variables, like semaphores, but using only a variable that is already needed when implementing things like a message queue, a ticket lock, a CLH lock etc.).

Other Linux improvements of that time were epoll (2003) as a poor man's equivalent of FreeBSD's kqueue/kevent (2000), libaio for asynchronous storage I/O (FreeBSD had used since 2000 an extended version of the POSIX AIO, adding "aio_waitcomplete"; libaio is better than the standard POSIX AIO), timerfd (2008) for waitable timers (FreeBSD had EVFILT_TIMER since 2000) and various means for doing zero-copy I/O, like O_DIRECT (2001) from IRIX, sendfile64 (2001), splice (2006) and related syscalls.

Of the BSDs, FreeBSD has the largest installed base, so it also has more developers and for the architectures that it supports (which are less than those supported by NetBSD and OpenBSD) it normally has the best performance among BSDs, as more optimization effort has been directed to it.

Before 2019, I would have said that for a server environment there was approximate feature parity between FreeBSD and Linux. But then Linux was greatly improved by the addition of io_uring, which finally provides the right way to do asynchronous I/O on Linux, where before this every available method was inferior in some respects to methods available on other operating systems 50 or even 60 years ago.

Now io_uring allows the implementation of high-performance I/O on Linux with much lower overhead than on any other UNIX-derived OS. Like previously with "futex" => "WaitOnAddress", the Microsoft developers remained able to recognize a good thing when they see it, so now Windows has also added functions like CreateIoRing (ioringapi.h).

While since io_uring I would always choose Linux for developing a server application where maximum performance is essential, I continue to use FreeBSD on some servers, where maximum reliability and maximum simplicity in management are more important.

I have FreeBSD servers that have been used 24/7 for more than 20 years, with reboots normally spaced at no less than a year, which were needed only for hardware upgrades or kernel upgrades. In the default configuration, or only with very minimal changes that can be done in a single configuration file, the BSDs are much more security-hardened than the default configurations of most Linux distributions. If you take your time to carefully audit and adjust all configuration options, you can reach equivalent security regardless of the operating system.

I have used occasionally NetBSD and OpenBSD, but not on general-purpose servers or PCs, which are better supported by Linux or FreeBSD. Especially NetBSD is easy to port to various embedded computers (which is of course an explicit goal for it).

First of all: The BSDs are ALL general purpose operating systems, not distributions. I just mention this because there is that general "OpenBSD is for routers" and "NetBSD is for old hardware" nonsense. To give you an example. Various things including 3D graphics acceleration for various devices has been available in OpenBSD and DragonFly a lot earlier than in eg. FreeBSD even though FreeBSD for example has proprietary NVIDIA drivers available unlike the rest. People use all of them on server, as desktops and even on gaming devices. It's even kind of a myth in BSD communities that "this BSD is for X".

Something that people who aren't in the BSD users already and especially ones that do use Linux sometimes misunderstand is that the BSDs are independent of one another. Eg. OpenBSD and FreeBSD are as different as eg. FreeBSD and macOS, which is also a BSD and has A LOT of kernel and user space level code shared with FreeBSD. That's why grep -v says FreeBSD that's why docker uses (or used) xhyve which is FreeBSDs bhyve for macOS, which is why you got kqueue and dtrace, etc.

So in that respect "the BSDs" doesn't really work that well and is "dangerously" oversimplifying things.

All of them have a pretty good security track with strengths and weaknesses compared to each other and Linux. I would say it's overall even. But depending whom you ask the difference may very, because even in the Linux world there are different approaches to security. And as anyone having anything serious to do with security it's not that easy to measure security in first place. OpenBSD has a strong focus and achieves a lot on the C side and people who don't use any BSD at all actually and basically don't know it have critical C code compile on OpenBSD to spot issues affecting their actual target platform.

As for development. All of the BSDs increase in terms of development (only happen to know it cause someone once pointed me at an obviously wrong graph). I got a bit interested in that comparison because something that people like to underestimate is that single individuals are still the ones doing huge chunks of big projects. It's insane. Linux kernel might be one of the very few exceptions because a huge amount of people make hardware essentially for it for a living (SBCs and so on). Even if you look at other "big" projects like Postgres and so on which huge companies absolutely rely on it's a couple of people doing insane amounts of work. You have that in all BSDs.

FreeBSD sees a lot of usage as a base for commercial OSs (video game consoles, Netflix cache servers, etc.). However to be fair while some parts come back to the OS they also have companies that shamelessly take code without giving pretty much anything back (looking at you, Apple). Nobody is angry at that cause that's the terms and people in the BSDs often contribute out of the drive to make a great OS.

Something in my opinion greatly overlooked is that FreeBSD has one of the best package bases of all open source operating systems including all the Linux distributions. This is somehow rarely advertised but you have a huge official, very stable, very up to date, highly configurable "repository". I think this is pretty much unmatched elsewhere. It's like having Arch AUR, but without the security nightmare nor instability. In fact getting a commit bit is on the harder side compared to many Linux distributions. Many official package maintainers cannot commit on their own. This means that the "many eyes" does a lot of work there.

You get binary packages with one of the best binary package managers out there (classic case of being late to the game, but taking all the leanings and doing a great implementation).

You get close to 35k of these "ports" both binary or compiled if you want. If you compile yourself you get a huge amount of flexibility. My go to example for this is pick and choose a Postgres and PostGIS version combination, but there are many other examples from disabling adding docs, to optimizations to a stunning amount of third party nginx patches giving you better than commercial version experience. That coupled with a great flexible automatic build tool called poudriere, which you can use to create repositories of all or parts of these 35k ports and your chosen options for different versions of FreeBSD.

I am not completely sure but the sheer amounted of vetted(!) packages is unseen anywhere else. There is a security database, there is deprecation warnings and since the base OS and package separation (maybe the common differentiator of BSDs compared to Linux) you can match stability with the latest packages (or older version, eg. in Postgres), which is something I really miss in the Linux world even though there is nothing that is because of the OS/kernel it runs. It's simply nobody in the Linux world did put the effort in so far.

While FreeBSD has had a couple of not so great defaults - I think maybe to do with how many long term FreeBSD users any ways have their custom configs and trying to be compatible with upgrades - I think it got better. On the other hand they have and had a lot of projects on that front. jails predated pretty much everything else in the open source world, there was the TrustedBSD project, now there is CherBSD which puts a capabilities system in to hardware, etc.

OpenBSD also has a strength that is overlooked in that for a lot of "set and forget" style applications the base OS is enough. You get a highly secure (custom) HTTP server, a custom acme client, a custom load balancer (relayd), one of the best firewall implementations, etc. out of the box. I love to use it for simple I need something so that one can upload files via SFTP to a HTTP server setups. Just run syspatch and every half year sysupgrade and you are good. OpenSMTPD is also an absolute breeze and also having a Go application running with pledge and unveil (basically a way simpler easy to use seccomp) is great. Also for some reason there is a serious gaming community on OpenBSD making things work without wine (doesn't even work on the OS) in a way that is hard to believe.

Another thing is that a lot of "everyone uses them" tools come out of OpenBSD circles. Eg. tmux, OpenSSH, sudo (even though doas replaces it), OpenNTPD, OpenBGPD femail, etc. Also LibreSSL and sndio which always pop up on Linux in the least expected places (like games) for some reason. They all of that heavy focus on simplicity, but done in a non-academical but very real world scenario. They also do a lot of work in the compiler scene on the security side.

NetBSD and DragonFly also have their often very unexpected strength. Eg. DragonFly supports multiple ways of full file system encryption, including truecrypt compatible one. NetBSD while working on a Dreamcast also has very interesting and it runs in unexpected places like the ISS and somehow there have always been DARPA contributions, while in all BSDs at least subjectively the most to NetBSD. Mostly because there I've also seen quite a few pkgsrc (their non-base-OS packages) contributions. pkgsrc probably is one of NetBSDs biggest impacts being using by lots of other OSs from Minix to illumos. And on that topic, Android has (had?) quite a few NetBSD parts (in place of GNU userspace software) and Microsoft took the network stack from the BSDs and Internix (which was the predocessor to the Linux subsystem from when they wanted to be compatible with Unix datacenters, etc.) was largely a modified OpenBSD system.

Again, not a BSD users, and BSD people being very practical and less of a cult they'll usually tell you to keep using Linux if it works for you. But I think there also are a lot of misinterpretations when you used Linux all your life.

In fact the hardest thing on just trying it will probably be it NOT being Linux and also when switching between BSDs. They are simply other approaches to making an operating system. So if you take away one thing from this it's they are general purpose operating systems in their own respect and while they share code so does macOS, which as mentioned is also a BSD.

Use Linux if it works for you, but just for people trying it out: If you wanna actually be able to compare it you really have to use it and get it else you will just be annoyed with it not being Linux. Kind of like when you try a different editor, programming language, etc. Just don't make the mistake thinking BSDs are like distributions. That's a bit like thinking AIX and macOS are just Unix distributions. Even Solaris I think in very old versions was BSD based. ;)

But ... if you want to teleport back to the 90s and go back to an era before Linux took off on a rocket ship to universal dominance then the BSD are for you. You will find a much more curated experience when people thought that documentation mattered. It is the road less traveled.

Anyone know why they are still on such an old version? The latest version is 16.1.

New x64 systems? Probably not. But since it still supports x86 and a whole slew of other architectures[1] still that don't always, it probably wouldn't make sense to stop.

[1]https://www.netbsd.org/ports/

NetBSD literally maintains support for VAX. Your mental model of expected hardware isn't exactly wrong in the general case, but it's also not exactly applicable here.

You can make ISO images that boot fine from both CD and USB sticks, when written to either format. Why "Fix" something that ain't broke?

CD-RW/DVD+RW on the other hand are also even worse than flash in that regard and decay very quickly in my experience.

I mean, you could also ask "why use NetBSD when Linux or even Windows is much more ubiquitous and much easier to install?", but I feel like that would be missing the point. Once you're off the beaten path, you have to update your priors a bit about how likely it is to need to support something that isn't as common.

Old Systems. I have one such system that will is in line to move to 11.0 from 9.3. To do that I will boot from CDROM. It does not have any USB ports.

NetBSD is probably the only UN*X Type OS that can run on many of these 32 bit systems. Plus the NetBSD team is doing their best to support these old systems.

That saves me $ and helps the environment a little bit :)

Graphics programming is very hard. Borland tried to improve the situation in the 90s with it graphics library, but little progress has been made since then. One of the most difficult things today is finding out the coordinates of the drawing area. /s

For starters, on the old BSDs there was no Almquist shell, unless you went to Usenet, grabbed the shar, and built it. The TENEX C shell was the hot new thing, as were BIND and Sendmail. And we got genuine Joy+Horton vi.

Whereas modern NetBSD does not have Sendmail any more, has nvi2, and supports access control lists. And of course under the covers has lots of things that the old BSDs didn't have, like UFS for TiB disc volumes and kqueue.

I've heard tell that they're even thinking of finally doing away with BIND in base.