In 2003, Egenera boxes could compile Linux in around 15 seconds.
Even my dual 7402 with 96 threads and 512 GiB of RAM can't compile a maximal Linux config x86 build in RAM in under 3 minutes.
What I find helps repeated builds is maintaining a memcache server instance for sccache.