Hi,
If you ever get to writing a blog post about kv cache quantisation, i'm interested in quantising K differently than V
If you like, you can run these tests yourself as well. It is ~$500 per a single combination, and assuming no failed runs.
If you like, you can run these tests yourself as well. It is ~$500 per a single combination, and assuming no failed runs.