Jepsen: ParticleDB 1.1.4

> ParticleDB 1.1.4 claimed to be a Postgres-compatible database ... Instead, we found a library of confusion

> exhibited frequent, profound transactional anomalies in essentially every way we could measure. We routinely observed the loss of committed writes, internal consistency violations, G1a (Aborted Read), G1b (Intermediate Read), and P4 (Lost Update). In the append workload, successfully appended values would routinely disappear from later reads of a given key. Our test suite inferred every kind of cyclic anomaly including G0, G1c, G-Single, and G2-item, but many of these can be attributed to more basic issues, like write loss. These issues occurred with a single server and without faults, and affected every transaction mode, interface, and WAL mode tested

Definitely seems worse than usual (looking back most databases seem to have <10 issues total, and the majority seem "mild") - since it's a database for AI was it also likely vibe-coded or just new? Then again even if new, I wouldn't expect so many issues in a stable release

I guess that's what happens when you vibecode a database...

It takes gumption to build a new storage engine. I remember when etcd[0] first made it's way into @aphyr's cross-hairs. We took all of the input very seriously because we knew that a data store which is untrustworthy is useless.

Resolving these issues likely won't be easy, but if you can get to the point where you walk away from Jepsen unscathed the trust in your software will be greatly amplified.

0: https://aphyr.com/posts/316-call-me-maybe-etcd-and-consul

Some software is correct by construction, in the sense that the overall design makes sense and the cases that don’t work are generally local bugs.

Some software is incorrect by construction, and it tends to work by some combination of luck and by carefully working around the incorrect design in most or all code paths that could expose the incorrectness. (AI can be quite effective at finding the relevant places to insert workarounds.)

ParticleDB sounds like the latter category. Plenty of large projects have portions that are in one category and portions that are in another.

As an aside, for the last day or two, I’ve been contemplating that the Linux kernel’s handling of folio pinning seems to be in the incorrect-by-construction category. Good times. No, I haven’t found a hard-to-fix bug that I would credibly characterize as a security or likely accidental corruption issue [0], but this seems mostly like luck.

[0] I do appear to have re-discovered a bug that some massive Anthropic dump from a couple months ago also found, and it’s getting fixed. If your application does “fixed” io_uring I/O or certain types of iommu-based or maybe RDMA on a memfd, you might plausibly encounter it on rare occasions. I didn’t bother making a reproducer.

Perhaps they meant formerly, probably just a speech to text problem

(Edit for clarity: this is quoted from the linked article)

  You could write predicates over non-existent columns:

  root=> SELECT * FROM t
         WHERE definitely_not_a_column = 4;
   a | b
  ---+----
   4 | 5
  (1 row)
  Or ask if 4 = 5

  root=> SELECT * FROM t WHERE 4 = 5;
   a | b
  ---+----
   5 | 6
  (1 row)

It must have been darkly amusing to see all these defect tickets coming in, only for them to frantically shovel each one to increasingly overworked LLM agents to apply pig-lipstick while the humans in charge oh-so-slowly start to realize their software quality is not quite what their glazing agents had been telling them and that what they have on their hands is basically sub-mongo-quality slop. What a world we live in, huh?

Especially now, where you could probably just ask AI to adapt the existing suites to the cosnistency claims this database makes, and get something useful out of it. While certainly not as good as a professional test, I'd expect it to at least find some low hanging fruits before going for the expensive and public official Jepsen test.

This is such a huge reputational hit, seems like a terrible outcome for a Series B company funded by big name VCs

What does this say about their main product?

"Written in Rust · Formally Verified · PostgreSQL Compatible"

EDIT: It's up now