My use-case is coding. I'm typically coding interactively with an agent most of the day, and then I have overnight jobs that run every night.

As for the infra setup, I am currently using llama.cpp, although I'm just now working on switching to vllm since it is better for concurrent inference, since it can interleave prompt-processing and decode. Until now llama.cpp was enough, since I was on an R9700 and it was only practical to run one agent with decent sized context anyway.