There’s no reason to believe that companies running open weights models would use consumer hardware. I can speak on behalf of my employer and say that we’d vastly prefer to have enterprise grade servers running inference on premises.
There’s no reason to believe that companies running open weights models would use consumer hardware. I can speak on behalf of my employer and say that we’d vastly prefer to have enterprise grade servers running inference on premises.