I have a 1500 dollar machine that can run it at 50 tok/s (3 V100s)
How did you buy 3 V100's for $1500??
Not OP and just guessing, but probably SXM2 GPU modules for the V100. Those can be acquired fairly inexpensively, but there is work to do to get them working together and the V100 has some limitations on the types of models you can run.
How did you buy 3 V100's for $1500??
Not OP and just guessing, but probably SXM2 GPU modules for the V100. Those can be acquired fairly inexpensively, but there is work to do to get them working together and the V100 has some limitations on the types of models you can run.