It would be great to have upstream llama.cpp support for this!

Agreed. They always sound exciting to try out but are such a pain to get working.

I always just throw an agent at it. Is this the RSI I keep hearing about

RSI saves you from RSI