> The whole point of discovering knowledge is to enjoy the process and share that with the world to create hope an optimisim in it, not to product like a factory.
This is the crux of everything you've written and I strongly disagree. The purpose of knowledge is the sheer power it brings, and if there's factory scale churning out of it, I'll gladly take it.
As a closer analogy, consider this probability (I'll be stealing from the game Stellaris here): let's say a few hundred years later, when we are spacefaring, we discover a vast cache/library of knowledge of an alien civilization, we won't just pass up on exploiting it because we need to "enjoy the process of acquiring it on our own" or "derive the satisfaction of sharing the discovery with others". No, I and I daresay most others will take as much as we can without restraint in a quest to make all our lives better/more enjoyable.
And if that's ok to do, using machine intelligence to mine/grab any new knowledge or new math as Tao is suggesting here should also be acceptable and welcomed.
> The purpose of knowledge is the sheer power it brings
A dismal prospect.
When knowledge is acquired at a human scale, and under the influence of a human culture, then extraordinary things can happen. Agriculture, the transistor, Gödel's theorems.
When knowledge is "churned out" at "factory scale" then human flourishing isn't even a by-product.
> This is the crux of everything you've written and I strongly disagree. The purpose of knowledge is the sheer power it brings, and if there's factory scale churning out of it, I'll gladly take it.
Actually, this is the fundamental divide. There are a group of people (like you) who want the world to be one way, and there are a group of people (like me) who think that world is abhorrent. I don't think we should go after that power.
But, I'm glad we got to this point in the discussion because it shows this irreconcilable divide. The truth that most people will never say is that these two groups are enemies and will forever be. They fit into this world because we currently participate in the same global system, but because of this situation, it's an unstable one. And AI is exactly the system that is distilling this antagonism to its limit.