GEFS on OpenBSD: A Early Preview

Posted by sippingabonedry 12 hours ago

Comments

Comment by yjftsjthsd-h 11 hours ago

> While snapshot consistency is useful to keep data consistent, disks often fail over time. In order to detect corruption, block pointers contain a hash of the data that they point at. If corrupted data is returned by the underlying storage medium, this is detected via block hashes. And if a programmer error causes the file system to write garbage to disk, this can often be caught early. The corruption is reported, and the damaged data may then be recovered from backups, RAID restoration, or some other means.

Okay! It's got CoW, snapshots, and data checksums. Therefore, it's good enough to compete with ZFS while being way smaller and permissively licensed. Now I just want it ported to Linux and the other BSDs:)

Comment by throw0101a 8 hours ago

> It's got CoW, snapshots, and data checksums. Therefore, it's good enough to compete with ZFS while being way smaller and permissively licensed.

Does it have a built-in RAID layer? Because if it doesn't, then it can't compete with ZFS in many use cases. For example, what does "data may then be recovered from […] RAID restoration" mean?

With ZFS, if you have a (e.g.) mirrored/RAID-1 configuration, and you fetch some data from one drive and the checksum is wrong, ZFS can check the other drive, and if that checksum is good it can (a) pass the good data up, and (b) use the good data to fix the bad data. Most mirroring systems can't do that both-drives checking: ZFS is self-healing.

(This isn't to say that GEFS won't be useful in many other situations.)

Comment by oridb 5 hours ago

Author here: correct, it's currently on the user to deal with corrupted data. I'm not opposed to adding a RAID layer, but I also want to explore what adding trim-style feedback to a generic RAID layer may look like, so that different file systems could share the work.

We'll see where things go.

Comment by throw0101a 4 hours ago

> […] but I also want to explore what adding trim-style feedback to a generic RAID layer may look like […]

TRIM commands sent to the block layer would probably help with SSD/NVMe wear leveling, as well as thin-provisioning in virtualized environments.

And the lack of RAID is not a critique: if you're mostly interested in the file system layer that's fine, and a useful area to solve problems in. My comment was more towards the comparison to ZFS and its "rampant layering violation" [1] and some of the trade-offs that happen when you focus on more or fewer parts of the stack.

[1] https://web.archive.org/web/20070602005153/http://blogs.sun....

Unrelated: and since you're surfing the comments, with regards to your superblocks [1], you may wish to look into some of the lessons learned by ZFS; specifically see perhaps Allan Jude's "Large Label and Blockpointer-V2" from the 2025 OpenZFS Dev Summit: https://www.youtube.com/watch?v=3tqOBv8BmXI

[1] https://orib.dev/gefs.pdf § 9.1

Comment by oridb 3 hours ago

I don't mean a literal trim command to the block device, but hints that get passed to the RAID layer to mark how blocks should be spread among the backing devices. I haven't put much thought into the details, but I suspect that there might not be so much information needed, so the RAID layer could be both generic and smart. It may not work out.

I'll probably be picking Allan's brain on a few things in the coming few months; amusingly, he mentioned doesn't use ZFS for ZFS development so that he can unload and reload the module, and he's interested in something that won't corrupt when he panics the kernel.

Comment by atmosx 11 hours ago

I don't think it will compete with ZFS or BTRFS (e.g. I don't think ppl will use GEFS over ZFS or BTRFS for a storage server), but it's a modern, much needed FFS replacement.

Comment by yjftsjthsd-h 10 hours ago

Who said anything about storage servers? I'm using zfs on laptops and desktops right now because I want data checksums and a filesystem that doesn't have a history of breaking horribly (I dropped btrfs after the second time it hosed my rootfs). Given the license issue with zfs - and in particular, the technical fallout like needing dkms - I'd be very pleased to replace it.

Comment by sippingabonedry 10 hours ago

> I dropped btrfs after the second time it hosed my rootfs

btrfs fans use the "you're using it wrong" excuse a lot.

I recall a failure mode that activated when you fill the FS to 100% and their response was "you should never fill a filesystem to capacity"

Comment by thetallguyyy 7 hours ago

It's really a shame, though. btrfs is permanently unfinished, and zfs has the wrong license. End result is people stick with ext4 or xfs most of the time anyway.

Comment by scheme271 6 hours ago

That failure mode is still there although it's been mitigated a bit. I hit it a few months ago but was able to recover after a few hours of doing various stuff.

Comment by sippingabonedry 6 hours ago

Yeah that's not how filesystems are supposed to work.

You should be able to dd /dev/zero to a file and not worry about the entire filesystem shitting the bed.

Comment by crest 4 hours ago

They also fucked up their parity RAID repair so that the next disk failure on a different disk will destroy the data, argued that the pathological B-tree performance reported by a user couldn't be real, because they didn't understood the tree depth would only be logarithmic to the base of the fanout if the entries are the same size. insert shocked pikachu face

Comment by jeffrallen 9 hours ago

Otoh, good luck bringing a CoW filesystem back from 100%. Delete a file? Sure, let me just make a copy of all the metadata that was pointing at it using... the zero blocks I have left.

Tradeoffs are a bitch, bitch.

Comment by yjftsjthsd-h 9 hours ago

Which is why ZFS reserves "slop space" to make sure that doesn't happen, instead of defaulting to making it easy for users to corner themselves like that.

Comment by whalesalad 10 hours ago

I have been hearing noise recently that btrfs is risky and unstable but (knocks on wood) i've been running it for years now with zero issues. What am I missing?

Comment by gucci-on-fleek 9 hours ago

btrfs is almost 20 years old now, so lots of people only used it back when it was newer and far buggier.

In my experience, btrfs is actually more reliable than other filesystems due to its checksumming abilities, but when it does fail, it's much harder to fix than with other filesystems (which will often try to continue on even when stuff is broken).

Comment by 4 hours ago

Comment by crabbone 9 hours ago

Failures in storage software are very rare. Which makes it very hard to test... (you need to run it a lot, for a very long time if you hope to find errors by chance).

Also, some failure modes are worse than others. The failures known as DI (data integrity) are the worst. Even though they aren't expected to happen to everyone at a certain frequency (because, again, mature storage software is comparatively very reliable), even a single DI error that happened to any user sets up a major alarm.

In the storage industry, the running joke is that after first DI in your product you lose funding, after the second DI you loose the product.

And it did happen to Btrfs quite a bit... I've seen it with my own eyes when a system didn't come back after power failure. (But I'm in the business of testing software storage products, so, it's less surprising that it happened to me).

So... it's perfectly plausible that you have never seen Btrfs fail, and it's been more error prone than eg. EXT4. The error rate is low enough so that if you don't actively try to cause the error you will never experience one. But, over a large group of diverse use patterns, the rate is still worse than expected.

Comment by atmosx 10 hours ago

> Who said anything about storage servers?

I did.

> I'm using zfs [...]

ZFS is primarily used on single-storage appliances.

Comment by gigatexal 7 hours ago

It’ll replace brtfs if it gets ported. Almost anything is better than that pile of garbage.

Comment by mmooss 10 hours ago

I've always wondered about similar designs: Doesn't calculating a hash of every block, on every read and every write, create lots of overhead? Why isn't that a problem?

Some systems have dedicated crypto co-processors for confidentiality (encryption) - e.g., I think drives with FDE, and I think Apple Silicon SoCs might have them. Can those be repurposed for hash calculation? What about systems that lack them?

Comment by chasil 10 hours ago

Both ZFS and modern btrfs support a large set of checksums.

Both implement sha256, which does impose a heavy speed penalty.

ZFS allows you to adjust the checksum on the fly, using something faster (Fletcher) if desired.

In btrfs, a global checksum is set at filesystem creation; xxhash is the best modern option.

There is a website: https://xxhash.com

Deduplication adds concerns for a strong hash free of collisions.

Comment by ThePowerOfFuet 8 hours ago

Fletcher has been the default for quite some time.

Comment by throw0101a 5 hours ago

Specifically "fletcher4":

* https://openzfs.github.io/openzfs-docs/man/master/7/zfsprops...

* https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Data...

* https://en.wikipedia.org/wiki/Fletcher%27s_checksum

* https://people.freebsd.org/~asomers/fletcher.pdf

* https://www.intel.com/content/www/us/en/developer/articles/t...

Originally documented in the paper "An Arithmetic Checksum for Serial Transmissions" (referenced also in RFC 1146, in the context of TCP).

Comment by yjftsjthsd-h 10 hours ago

Yes, it adds some overhead, but it's fine IME. Granted, it helps that compression can significantly speed up performance. (I was very confused the first time I saw ZFS reading data faster than its drives were physically capable of, because it turned out the CPU could decompress faster than the drives could read)

Comment by crabbone 9 hours ago

Programs like filesystems typically have their own schedulers that aggregate writes, it would be an extreme performance hit if they wrote each checksum individually, as a separate I/O operation. They are certainly bundled with some other data that needs to be written.

And if you are concerned about the compute rather than storage, then writing to a block device is still slow enough so that computing a checksum isn't important performance-wise.

Comment by g0xA52A2A 12 hours ago

There was a recent presentation on this at EuroBSDCon for those interested.

https://events.eurobsdcon.org/2026/talk/NVMSCJ/

https://exquisite.tube/w/3QQimMdswWJxrsPaJtak2u

Comment by limagnolia 6 hours ago

I've always wondered why HAMMER2 from DragonFlyBSD hasn't gotten more attention from other OSes?

Comment by rzerowan 6 hours ago

I think its the way their kernel architecture has diverged to this point in time.Seeing as it depends on structures/primitives in DRagonfly that dont exist in the other BSDs. Making a port a bit of a challenge.

Comment by tiffanyh 8 hours ago

All respect to the author on their work, but for me personally - I'm be more interested in Hammer2 (from Dragonfly BSD) to come to OpenBSD.

https://github.com/kusumi/openbsd_hammer2

Comment by moody__ 11 hours ago

I've been following (and helping test) gefs on 9front for a while now. 9front's nightly builder has been running off of it for quite a while. Ori's done a fantastic job.

Comment by sellmesoap 8 hours ago

Oh my gosh, while I may have heard of 9front in passing I had a little look today and it's super cool! also shithub what an amazing alternative forge, it feels like one of those days when I'm part of the 10,000!

Comment by dchest 12 hours ago

Talk at EuroBSDCon 2026: https://exquisite.tube/w/3QQimMdswWJxrsPaJtak2u

Comment by sippingabonedry 12 hours ago

GEFS: A Good Enough File System https://orib.dev/gefs.pdf

Comment by throw0101a 8 hours ago

From NYC*BUG† May 2023, "GEFS, A Good Enough File System" By Ori Bernstein:

* https://www.youtube.com/watch?v=juFndFy72gI

September 2026 EuroBSDCon presentation from Sunday:

* https://www.youtube.com/watch?v=yPoU4QEv_u8&t=49m43s

† BSD User Group

Comment by fn-mote 11 hours ago

Is there any chance of proving a filesystem is correct?

Is this one simple enough that it won’t have bugs??

Given the issues with well-known filesystems like ZFS and BetterFS, why shouldn’t I expect data-losing bugs in this one?

Comment by spijdar 11 hours ago

I think the premise is basically yes, you should assume there will be data-losing bugs, but:

1. The filesystem should be reasonably good at detecting an error/corruption state and informing you, and

2. You should have backups of said data stored elsewhere, and backups should be tested (e.g. to verify that data can be read back)

Comment by cyberpunk 11 hours ago

What well known data-losing bugs are there in zfs? It can be slow, and resource hungry, but afaik it's about as safe as they come (and I've been using it in prod since solaris 10)

Comment by alethic 11 hours ago

There was a long-running data corruption issue with non-raw sends that was finally found and patched in 2025: https://github.com/openzfs/openzfs-docs/issues/494. But I agree, it's about as safe as they come. I trust it far more than any other file system, in large part due to all its built-in redundancy and the way it makes backups trivial (encrypted sends <3)...

Comment by sellmesoap 11 hours ago

I never dug into the failure, but I once had a ZFS get to a state where it would crash the kernel on mount, I was able to mount it with checks turned off an recover what was important, but it was a spooky experience. I live on the bleeding edge of file systems for my desktop, I was on reiser4 back when that was fresh, I daily drive bcachefs (it's been great!) Mostly I've been lucky, I don't usually keep an openbsd system around, but I love FreeBSD and I'll give OpenBSD a try with GEFS for sure!

Comment by yjftsjthsd-h 10 hours ago

Its native encryption has something of a poor history

Comment by ThePowerOfFuet 8 hours ago

>Error handling is largely commented out.

First we do the first 90%, and then we do the last 90%.

Comment by calvinmorrison 11 hours ago

I have been running GEFS for a number of days and it hasnt crashed

248 ├gefs [ctl.1]

249 ├gefs [mutate.2]

250 ├gefs [sweep.3]

251 ├gefs [tasks.-1]

252 ├gefs [readio.4]

253 ├gefs [syncio.5]

254 ├gefs [srvio.-1]

255 ├gefs [stdio.-1]

up 13 days, 15:34:25

send it to production!!

Comment by geoffbp 11 hours ago

> The git repo is hidden on shithub

Heh :)

Comment by irusensei 9 hours ago

Is this classic 9front tomfoolery?

Comment by MisterTea 8 hours ago

Indeed. If that URL is too crude then one may also use only9fans.com.

Comment by doublepg23 12 hours ago

This is great timing considering I'm currently dealing with FFS corruption after my OpenBSD server lost power during a storm.

Comment by rbc 7 hours ago

I've run OpenBSD continuously at home without backup power for something like eight years. They probably loose power about once or twice a quarter. I have yet to detect lost data that was important. Measures are taken. If a storm is coming through, I'll shutdown till it passes.

I've been lucky with release driver support. The little Lenovo ThinkCentre's being used seem to chug along without crashing on driver issues, at least with OpenBSD releases.

The standard OpenBSD partitioning scheme is also being used. Boot time fsck has never failed when the storage was properly attached and in a good state. Backups are performed using pax. So far, so good.

Comment by daneel_w 11 hours ago

Are you sure it's not just a dodgy SSD that simply failed to persist data when losing power mid-write? I've had my share of sudden power cuts upon OpenBSD during the past 20+ years, and FFS has so far never gone corrupt on me.

Comment by doublepg23 10 hours ago

I do not believe so?

Drive is a 2TB Intel 670p NVMe SSD (INTEL SSDPEKNU020TZ) with 9078 power on hours and 42TBW - so pretty spry, but not at the start of the bathtub curve either.

It was mounted as fast storage for a Bitcoin node.

Perhaps the only 'unique' thing is it is using a NVMe to PCIe adapter card (Synology M2D20) due to this being my "legacy" server that's still rocking a Broadwell chip.

Comment by fodkodrasz 8 hours ago

FFS corrupted multiple times on me, with intel video driver freezing on OpenBSD. My short OpenBSD sidetrack ended after reliably corrupting itself the third time... every time video driver panicking, leaving a corrupted filesystem after reboot, when I had eg. ports install going on during panic.

Windows ran fine on the machine (Lenovo 200) before, and Linux ran fine after. FFS (and the intel video drivers) are the weakest part of OpenBSD in my experience, I liked many other aspects.

Comment by yellowapple 8 hours ago

I've seen file corruption with FFS on some of my OpenBSD servers, though them being VMs is likely a factor there.

Comment by oridb 5 hours ago

Qcow2 will zero blocks on power loss, at least in some configurations.

Comment by BoingBoomTschak 11 hours ago

What a wonderful surprise! The nearest thing seem to be modern (v5) XFS + dm-integrity, I'll have to see a comparison once it's stable enough.

A thing ZFS suffers from is fragmentation (no way to defragment in-place nor preallocate so stuff like bittorrent doesn't play well with it), which it justifies with its CoW design, wonder if/how it mitigates the problem.

Comment by kjs3 10 hours ago

If we're looking at 'future' filesystems, is fragmentation really an issue in an SSD world? Not that there isn't a lot of spinning rust (and will be for quite a while), I don't think it's unreasonable to assume "most block storage is going to be SSD in the future" when allocating resources to priorities.

Comment by BoingBoomTschak 9 hours ago

The day I can furnish my NAS in SSDs for roughly the same price as HDDs probably won't come before any current filesystem is obsoleted for some reason or another, methinks.

Comment by kjs3 7 hours ago

Probably true. My bad.

Comment by 6d6b73 7 hours ago

Fragmentation in ZFS wastes a lot of space. So it doesnt matter if it's SSD or not.

Comment by kjs3 7 hours ago

You're right...I didn't think that through.

Comment by jijji 6 hours ago

> Error handling is largely commented out.

I guess we won't really know when the file system breaks or corrupts data

Comment by metalforever 12 hours ago

Finally . Very good work team !

Comment by anthk 11 hours ago

Ori B. it's a great programmer, he fixed a small bug on the earlier GeFS on 9front versions in no time. It worked fine in my n270 based Atom netbook under 9front, so it will run perfectly well under OpenBSD in a near future.

It isn't as resource heavy as ZFS, and it will be more reliable than FFS, for sure.