Quickly reading the article, one notable limitation seems to be that these checkpoints are 512-1024 tokens context size models, while Jev is seemingly 32k.
That's a pretty big limitation, I would argue, unless I'm misunderstanding and it can be worked around easily somehow? I'm surprised it isn't surfaced more prominently in the comparison.
Jev has 64k total token request budget and I do wonder how it will handle highly specialised inputs.
This Jev waitlist that Typesafe AI are utilising is surely going to raise questions pretty soon - it's hard to sell this to bosses when it looks like a pop-up restaurant
It's already on Openrouter
It's difficult to get 3rd party gateway approval.
And on Vercel AI gateway
I just got my invite so the waitlist doesn't seem to be particularly long
I am more curious about a 60k prompt... I haven't seen much discussion about large prompts, is it still < 500ms?
Yeah, it's weird, considering ModernBERT, which the Laya models are based on, supports 8192 context window.