16 tokens / s is not nothing.
It's the other way around due to poor framing, it would be much easier to compare if you [the repo] said 0.02 tps.
Confusing numbers everywhere.
16tk/s... Then 3 tks per minute. Then someone else posted 0.3tk/s.
? Readme says 60-70s per token
It's the other way around due to poor framing, it would be much easier to compare if you [the repo] said 0.02 tps.
Confusing numbers everywhere.
16tk/s... Then 3 tks per minute. Then someone else posted 0.3tk/s.
? Readme says 60-70s per token