I'm a 6502 guy myself, but the Z80 instruction set is also easily holdable in one's memory.

As others said, a 4MHz Z80 is clearly capable of outperforming a 1MHz 6502 as is evidenced by the many ZX Spectrum demos that show off 3D/plotting effects.