Without knowing anything about your benchmark, it might be your harness at fault here. I say this because with longer thinking you run the risk of filling up your context faster and you need a good context compaction strategy in your harness to mitigate that.
I have heard good things about Pi which supports auto-compaction, but I can't personally vouch for it since I use my own.
You're probably partly right (though I use the same context size for all models, so there still is a difference in the model/quantization itself), but I definitely see it "thinking itself into an extremely expensive and error-prone approach without backing out". A common "failure mode" of my benchmark with <Q6 is "okay, I will implement a simulator", and when I see that I think to myself "please, no, you don't need to do that at all, you just missed a tiny thing" (but if I would type that into the session, it would void the benchmark; the idea is that I wouldn't know the solution myself after all).
Especially because Qwen3.8 27B does not seem to be good enough to implement the simulator with all its intricacies and tiny subtleties correctly. So it's churning for hours with no real progress, where it just had to explore the initial problem space a tiny bit more.
And very interestingly, even with Q6, turning off reasoning entirely does not make it fall into that trap, or many others, and it more often solves the task in record time. Both because of what I just described, but also because the endless reasoning costs a lot of token, which significantly translates to time on a home rig...
I see, maybe your use case is indeed pathological for Qwen 3.8 27b. Another thing to try if you are using llama.cpp is "reasoning budget". That makes the thinking stop after the budget has been reached and inserts a custom message you can choose, so something like "you have thought for too long, now continue with the execution ..."
For what it's worth, if you haven't already, you can also let it run overnight (if you have compaction enabled) to see if it ever gets out of that hole. The reason `xhigh` is the default is because 3.8 is trying to optimize for long horizon tasks where monitoring its every single thinking misstep might not be a good use of our time.