these were tests done on pre release builds and some experimental concurrency modes as well.... The engine has since been redesigned and all of these issues have been fixed..
Its also faster than Redis for KV, also numbers show this should be top on clickbench, official submission is in progress
Yeah, but none of those builds should have seen the light of day, given all the problems that were found.
And given the bug churn, something is very wrong with the testing harness. Either there was grossly insufficient testing to start with (indicating the humans involved were not taking the job of building a database server seriously enough), or else the LLM agents were mangling the tests to get the tests green while breaking the code (indicating that there was either no human review or the human reviewers didn't know enough about how the test+code ran to be able to catch the LLMs misbehaving).
What I fear is that the humans will implement more and more layers of LLMs to monitor the LLMs, rather than slow down development and learn what their code actually does.
Why should anyone believe that? I mean it might be true, I don't know. But if such a report turns out many fundamental issues it doesn't quite sound plausible that a redesign would suddenly not have any issues anymore. It might have fewer issues, it likely will have different issues as well though.
Because this is why companies pay for jepsen tests. Not because they enjoy to be destroyed in jepsen report, but to fix the issues.
This is like paying for a pentest of a treehouse, and claiming that this implies the updated treehouse with a working padlock instead of a novelty lock is in the same league as a bank or a data center.
It makes mongodb from the report a decade ago look good.
I'm not saying it is proof it is done. But most likely they are working on it
There's no prize for participation, and they very clearly didn't do even the most rudimentary testing before paying for an in-depth third party analysis. I'd say that this reflects a profoundly childish approach to software, but that is unfair to children: I remember how much testing I did of my crappy game before I submitted it to STFormat when I was 12.
They clearly aren’t the right team to be working on it. It’s best if someone else gives it a go.
Most Jepsen tests leave me with the impression "databases are damn complicated and difficult to get right". My thoughts with this particular report are "I'll never ever touch this DB if I can help it".
[dead]
> these were tests done on pre release builds
That's a convenient excuse, because now the website calls them "release candidate" builds, but as far as I can tell there was no such disclaimer back in May. As mentioned in the report, the Docker images were specifically advertised as being production-ready.
https://web.archive.org/web/20260518201315/https://particled...
https://web.archive.org/web/20260518200305/https://particled...
Ok sure, so an LLM told you all the bugs are fixed and it’s faster than Redis, just like how it told you before that it was formally verified?
Please try to resist the urge of being a meat proxy.
> before that it was formally verified
Jepsen is a fantastic tool - and one every database engineering team should have in their arsenal - but it proves the presence of bugs, not the absence of them. It is not formal verification, just a more rigorous form of existential testing.
I was referring to this claim from before the Jepsen report, which they have of course now quietly removed:
> ParticleDB’s home page advertised that ParticleDB was “formally proven” with a TLA+ proof. However, during our collaboration no TLA+ specification was available.
Makes you wonder if they asked their agent to formally prove it with TLA+ and it just responded that it had performed the proofing.
/dev/null is also an extremely fast KV store.
Sure, it still has some bugs resulting in occasional data loss, but it is pre-release so that's okay. Look at the speed, people!
You just need to raise 50m to crack that pesky "consistent read after write" nut!
The fact that they believed this product was anywhere near ready for a Jepsen test means they lack the judgement and engineering competence to credibly provide a database server. It doesn’t matter what their clanker does after they get the report and the tooling - they should have already had coverage for the vast majority of these issues and the fact that they now conform to a test doesn’t mean there aren’t MANY other product issues that will be discovered at the cost of their customers data.
Bro you've got to be kidding. pre-release builds? Aphyr directly addresses this in this analysis. Furthermore, given the absolutely ridiculous nature of some of these bugs, and the low confidence exhibited by the authors in the fixes, how can anyone assume that suddenly things are correct? The database was so janky that it sounds like most of the time was spent on basic bugs that a decent set of unit-tests would've caught. Who knows what wasn't uncovered because the db was so broken to begin with?
I mean did you see the "primary key equality" thing? Holy shit.
I just wonder why, from all the things you can vibecode, you would pick a database if you don't know the first thing about database design?
I look forward to the second report from Jepsen for the new implementation then.
> and all of these issues have been fixed..
You can't be sure until they test it again
> Its also faster than Redis for KV
No. Wrong. False. All performance numbers are plainly invalid. As my coworker says, performance numbers don't count if the code is broken.
> Hey Claude, could you find some releases we can send to Aphyr for testing?
... executing SELECT * FROM releases WHERE 4 = 5 -- it's not just a slopdb, but TLA+ load bearing and 100% Jepsen ready
ParticleDB 1.1.4 ParticleDB 1.1.18.11.rc-22
2 rows returned
https://github.com/ClickHouse/ClickBench/pull/2952
Previous attempts were strange, to say the least: https://github.com/ClickHouse/ClickBench/pulls?q=is%3Apr+par...
Even the PR description reads like it has been generated by an LLM.
What a waste of CO₂.