I just used it on a Apple M4 MacBook Pro with 48GB RAM with llama.cpp and Pi to help diagnose an infinite looping request in a React Server component on a Next.js application.
After about 10+ hours of digging, it has apparently found a bug in the Next.js framework, with an example app that replicates the bug, and a fix for now to disable prefetch in the Link component.
I had in my prompt asked it to discover the root cause of the bug and propose a fix, but I did not expect it to dig this deep.
I am eyeing one of these specifically for this use case, could you please post roughly what kind of tokens per second numbers you get for text generation for this 27B model?
edit: and which quant you are using, please :-)
10 hours at what tokens per sec?