new | ask | show | jobs
Tepix 6 hours ago  [ - ]

Sounds good. I saw that you use the FP8 version of the model. Do you also quantize the KV cache?

sacrelege 4 hours ago  [ - ]

no I don't, since there seem to be a silent degradation bug