> Sadly until china scales production of hardware it really isn’t economical to run this stuff yourself.

I'm running this stuff at home on my desktop and using it through an app on my phone. 60-140TPS depending on model / use case.

It's more than fast enough to even maintain voice conversation.