This doc on Ilya Sutskever & Geoffrey Hinton gives a great background on the progression of deep learning over the past decades [0].
Tl;dr - compute was the bottleneck.
I am not associated with this channel/video, just love it. I’ve shared it here before.