Disk can lie to you when you write to it
Posted by jtregunna 1 day ago
Comments
Comment by jmpman 12 hours ago
I’ve seen disks do off track writes, dropped writes due to write channel failures, and dropped writes due to the media having been literally scrubbed off the platter previously. You need LBA seeded CRC to catch these failures along with a number of other checks. I get excited when people write about this in the industry. They’re extremely interesting failure modes that I’ve been lucky enough to have been exposed to, at volume, for a large fraction of my career.
Comment by compressedgas 21 hours ago
I thought an fsync on the containing directories of each of the logs was needed to ensure the that newly created files were durably present in the directories.
Comment by jtregunna 15 hours ago
Right, you do need to fsync when creating new files to ensure the directory entry is durable. However, WAL files are typically created once and then appended to for their lifetime, so the directory fsync is only needed at file creation time, not during normal operations.
Comment by breakingcups 14 hours ago
> Conclusion
> A production-grade WAL isn't just code, it's a contract.
I hate that I'm now suspicious of this formulation.
Comment by jtregunna 13 hours ago
In what sense? The phrasing is just a generalization, production-grade anything needs consideration of the needs and goals of the project.
Comment by rogerrogerr 12 hours ago
“<x> isn’t just <y>, it’s <z>” is an AI smell.