Beyond the missing latency and compute comparisons that Heaney commenter mentioned, also nothing about its error rate compared to Jev (nor if it even always outputs in a format the app can parse, not sure how solved that is).
But then at the end it says it’s parody. Maybe HN title should say it’s a joke.
The parody note at the end is bizarre. Everything above it reads pretty seriously:
> Everyone on Twitter is all over Jev, how it's the next frontier of large language models and the AI paradigm. We don’t really think so.
latency and compute comparisons highly depends on your local setup.
you can swith to a better model for lower error rate.
Which massively slows down the output. Doing this with Qwen 9B already takes you into seconds per answer territory, and Jev is supposedly frontier level intelligence.
Yeah it says it's a parody, but then in the same sentence it refers to the other "OpenJev" implementations, which are basically the same thing with marginally more effort. And it doesn't imply that those things are parodies too (and I don't think they are parodies).
Somehow the HN crowd has a bunch of "professionals" who don't care about error rates and think that a Qwen model running on a potato is frontier intelligence.
1) get local model to run on the electrical output of a potato 2) accept Nobel price
You didn't specify time frames; 1) is doable for a very short time, with a lot of coulomb caching in between the computer and the potato :).
(For more realistic solution, surely someone must be working on optronics - these models just beg to have their weights cleverly etched into stacked sheets of plastic, so they can do inference for free on a beam of light.)
> nothing about its...
Non deterministic systems have furthered the "brain rot" in our industry.
Lots of people were happy to ignore the code in their "supply chain" before LLM's - but suddenly not reading the LLM's output is a problem. I get they are different but we're in the same realm.
The lack of real data on performance of what ever application that one is trying to pitch is getting appalling. It's a lot of "trust me bro" this works better hand waving. And it's getting gross.
And how do we even measure nondeterministic systems? Because if I told you that Anthropic was spending millions of dollars having 1000's of agents "pre solve" benchmarks to build into their next version of the system you would scream they were cheating. Every one is focused on the "hacking" in the hugging face incident and no one is looking why they were even playing with those benchmarks in the first place.
"Trust me Bro"...