> It's not like CPUs are slow, in fact they're quite a bit faster than any single GPU thread.

This was overwhelmingly true ten years ago, not so much now.

Modern GPU threads are about 3Ghz, CPUs are still slightly faster in theory but the larger amounts of local fast memory makes GPU threads pretty competitive in practice.