they just released the first small models that i would consider even vaguely articulate for edge inference involving a human. maybe they want to do a mistral and raise a kajillion and work from their home town?
they just released the first small models that i would consider even vaguely articulate for edge inference involving a human. maybe they want to do a mistral and raise a kajillion and work from their home town?
What does do a mistral mean?
MistralAI is known for their smaller models on the edge, to avoid competing with Gemini & OpenAI directly.
Who knows if OpenAI will do a refresh, but gpt-oss-20B/120B are still some of the best edge models so far.
oh?! what do they handle well? how do they fail?
the 3.5 9b model on my laptop at full fp8 is outlandish in its seeming reasoning capacity, though i haven’t really stress tested it