You can't have paradigm shifts by following the paradigm.
How I think of it is we need a distribution of people (shaped like a power law, not a normal).
Most people should be in the main body, doing what most people do. They're probably the "most productive".
Then you have people in the mid tail who innovate but it's incremental and not very novel. They produce frequently (our current research paradigm optimizes for this). But there aren't leaps and bounds. Critically it keeps pushing things forward, refining and improving.
But then there's those in the long tail. They fail most of the time and are the "least productive". Sometimes never doing anything of note their entire lives. But these are also the people that change the world in much bigger ways. And sometimes those that appeared to do nothing have their value found decades or centuries later.
Not everyone needs to be Newton/Leibniz. Not everyone should be. But that kind of work is critical to advancing our knowledge and wealth as a species. The problem is it is often indistinguishable from wasting time. But I'm willing to bet that the work of Newton alone has created more value to all of human civilization than every failed long tail person has cost us.
In any investment strategy you benefit from having high risk investments. Most lose you money but the ones that win reward you with much more than you lost. I'm not sure why this is so well known in the investment world but controversial in the research/academic/innovation world.
You would likely enjoy Isaac Asimov's "Profession": https://www.abelard.org/asimov.php
Absolutely love that book and have even mentioned it on here before too lol.
I highly recommend it. I'm also just a huge Asimov fan
I don't understand how this is a power law and not normal. The "long tail" is usually mentioned in a normal distribution being the right-most end of it.
Hah. I think of it as a slime mold. There's the main body (bodies?), but it's always shooting out little bits of itself that try weird stuff - founding underwater communes, or climbing mountains in Crocs or something. Most of these offshoots don't have that much of an impact, but occasionally one lucks out and discovers America or peanut butter and the main body saunters off that way.
Yeah we find this type of optimization all over nature. Even radiation is important to create noise. We need it in machine learning. A noisy optimizer is critical for generalized learning. Too much noise and you learn nothing but no noise and you only memorize. So there's a balance
I get the "outliers are useful" thing you're trying to emphasis. But as someone from a mountainous country, please dont "climb[ing] mountains in Crocs", we regularily get media reports of hopelessly underequipped people having to be rescued with a whole team of people, in the middle of the night, with horrible weather, usually also endangering the people that do the rescue. I guess what I am trying to say is, there is a limit to how silly you can/should be.
What is the expected value, of some dude at university spinning dinner plates in the cafeteria? What a silly pointless thing to do! Of course, if you're physics Professor Feynman, you get a Nobel out of it, so do the silly pointless things after all!
nice observation.