> why people are excited about Jev-style decision models.
I think its a combination of being new to classification models and not using in anger.
LLM Decision models are still relatively new and suffer from a number of issues. Order of choices impacts probability, it starts to break down over 20 classifications and even how you word things impacts probability.
> and not using in anger
Is this a typo? are you saying that this is so new people haven't struggled with Jev enough yet? very interesting
> it starts to break down over 20 classifications
Yes, I recently ran a benchmark on exactly that: https://jeffyclassify.com/blog/when-llms-fail-at-classificat...
At around 4 classes already LLM-based classifiers (Jev/Laya/GLiNER et al) start falling behind
> Is this a typo?
My understanding of the statement is trying to use it in production for the first time. Once you start to actually use decision models beyond something simple you start to see the limitations.