great to hear, which quant + KV quant + KV size are you on? is it llama.cpp?
Here's the writeup from the experiments that claude did: https://claude.ai/artifact/X6WJVtKdL1LrhYuCMqdXvG#b9a4d3e2-8...
Here's the writeup from the experiments that claude did: https://claude.ai/artifact/X6WJVtKdL1LrhYuCMqdXvG#b9a4d3e2-8...