Agreed, this is crazy, and is exactly why the administration did what it did. However, every frontier model will require this within months.
Agreed, this is crazy, and is exactly why the administration did what it did. However, every frontier model will require this within months.
You're right. I want to say that openai is a viable alternative, but they're even less trustworthy.
It might be time for me to start looking into Chinese models or purchasing hardware for local llms, even if the cost amounts to 5-10k.
For 15k you could run the best open source models (that would require 200k in hardware to run) 24/7 at 50 tps for like 5 years straight.
Im really not sure why anyone would spend 15k to run local llms. The models you'll be able to run (70b) param models will be incredibly underwhelming.
> For 15k you could run the best open source models (that would require 200k in hardware to run) 24/7 at 50 tps for like 5 years straight.
Is this assuming no price increase and no throttling ?