I’d like a “Bonsai 2.8T.” That is, something that is near the Fable/Sol/K3 class, but capable of running locally on consumer hardware.
I’d like a “Bonsai 2.8T.” That is, something that is near the Fable/Sol/K3 class, but capable of running locally on consumer hardware.
At 1.5 bits per weight it'll still be over 500gb - that's still not running on consumer hardware.
Best case they release smaller models. 120b class of qwen 3.8 would be incredible - it fits on device for those serious about AI, but without millions of dollars in hardware for terabytes of VRAM