Looking at the example Jev use cases, it almost feels like Jev's incredible cost/task can make it competitive as a generalized "poor man's ranking" algorithm that can be useful for lean startups or any fast paced development org.
I need to rank 1000 articles and pick the 5 most relevant for the user? Jev.
I need to audit and strip out content because my user is affected by regional privacy laws (without hallucinating)? Jev.
I need to surface the 3 funniest media comments that match the user's sense of humour? Jev.
More like:
I need to ...? -> Open-weight model.
I'm sure someones working on this as we speak using an open-weight LLM base (Qwen or something would be a perfect fit).
This sort of task is a perfect fit for a very small model capable of semantic parsing. You can get away with a LOT less parameters without all the autoregressive generation and long-context reasoning.
you don't say - https://huggingface.co/harshatheg/Qwen-2.5-1B-RLCD
Crazy, looks like this was just published a few hours after the TypeSafe post!
Wonder if this could lead to better recommendation algorithms.