Juergen Schmidhuber and his students came up with much of the basic NN elements, such as RNN, LSTM and others without behemoth resources (and several years earlier). Ttbomk, the one thing they DIDN’T come up with was the transformer; however it is very likely someone would have within a 5-10 year time frame.
I well aware of Schmidhuber, still the scaling of the compute was critical. The reason Schmidhuber didnt go all the way is still scaling/capital, which accrues to monopolies who can afford wildly speculative research. Also, LSTM, RNN, etc, while effective for their time, were dead ends.
I'm not saying that, I'm saying you need a company with huge amounts of cash on hand to spend on R&D. The monopoly provides a safe space for it, like it or not, this system has worked.
Juergen Schmidhuber and his students came up with much of the basic NN elements, such as RNN, LSTM and others without behemoth resources (and several years earlier). Ttbomk, the one thing they DIDN’T come up with was the transformer; however it is very likely someone would have within a 5-10 year time frame.
I well aware of Schmidhuber, still the scaling of the compute was critical. The reason Schmidhuber didnt go all the way is still scaling/capital, which accrues to monopolies who can afford wildly speculative research. Also, LSTM, RNN, etc, while effective for their time, were dead ends.
compute was critical, but compute was happening anyway. I fail to say why you are convinced that there would not be transformers.
I'm not saying that, I'm saying you need a company with huge amounts of cash on hand to spend on R&D. The monopoly provides a safe space for it, like it or not, this system has worked.