A significant amount of the effect is everything outside the proof of work. Not the cost of hashing but the need to run the javascript that submits it.
If you've seen anyone post a comparison of crawl rate versus difficulty, I'd love to see it. There's probably some difference but I want to know how much of the overall effect it is.