Great take and useful for the thought experiment.
I just feel like your taking it all as a given/fact. And maybe you’re right, but I’m not that well versed into it to say things for sure.
What I'm trying to get at is that if we’re considering a black swan event, why not consider both the bull and the bear case or even the neutral one and at least accept that we don’t know which one it will be and it may well be a coin toss…
Perhaps using “cancer” deviated things into a specific subject, but it could be anything. A drug discovery, quantum computing, etc.
Again, I’m no quantum physicist or chemist, but as a world citizen, these feel within the realm of possibilities. Just like it destroying humanity or simply just being yet another technological advancement like the computer or the internet was.
—-
I guess the people working on computers could have argued there was a 10% chance it was going to end humanity and so the people working on the internet because AI was going to come about.
—
I’d say that my final conclusion is that we don’t really know and the signals we’re getting from founders, investors, the market, here, and other people we talk to is that NOBODY KNOWS. All we can do is to continue to speculate and make bets based on hunches.
Yeah. What I think is deceptive is something like solving Navier-Stokes with $10M of compute. The bigger question is how many conjectures and hypotheses they tried, and what fraction of those actually succeeded. If they expended say $1B on various open problems, and only 1% succeeded, is that the kind of investment that one will make? (The $10M alone is more than what would have been devoted to academics investigating these issues. But realistically one has to quantify the cost of the entire enterprise, and the probability of success vs failure.
From OAI we're hearing only about 700+ successes. Relative to what failure rate? The cost of success isn't merely the cost of each success, but rather the overall cost of all experiments, of which -- I'd guess -- a huge fraction failed.
I think that’s what VCs do. Only a fraction of investments succeed and the ones that do pay off big time.
Now, imagine having a mechanism that will constantly improve, so you can go from 1% success rate to 5% success rate in say a year.
I see your point. What I think could be different is the speed at which things can be done compared to conventional ways of discovery.
Think of a COVID/MRNA like solution for other areas.
I'm all in on that, and I own Moderna stock ... so there's that. But we sort of understand the pathway there: we want to introduce a protein that induces an immune system reaction. Traditionally you re-engineer a virus that produces that protein that produces the RNA that gets transcribed into that protein. But instead, with better tech, you introduce that RNA directly. Simpler and faster. And perhaps you can use that mechanism to introduce other proteins for other ends (as recently with its personalized melanoma treatments).
I'm all for the notion that ML can improve 1% to 5%. But I haven't seen anyone suggesting HOW that happens. So far, it looks like taking 1000s of candidates to 10,000 candidates, which just further clogs the pipeline. Using ML to automate lab tests seems like a win there. But still, pushing those through phase 1, phase 2, phase 3 clinical trials is the bottleneck. 90% of what enters that pipeline dies, and each involves many years and many $100s of millions of investment.
But as I noted: the quicker path would seem to be through peptides rather than small drug molecules. ML does pretty well with proteins/peptide chains and mRNA provides a way to introduce them (or injection, like GLP1's). Still requires all the testing, though, but presumably we know more about how they degrade, etc. over time.
It seems you just don't wanna see the bubble. Or maybe there isn't a bubble and that's your call to invest more? Amodei would be happy.
Oh, I see it. I’m just thinking 5–10 years down the line not short-term volatility.
Could there be more ChatGPT moments down the line in other areas like science, genetics, space and something we haven’t considered? I think there’s a chance.
I also think AI is here to stay and there may be some confirmation bias from the community regarding open source models. It’s like someone saying back then that Linux was going to take over… (?) Open source software seems to be served for technical people. Not for the average Joe looking for an answer or help generating a social media post or help writing an email.
Indeed, ML is here to stay. But there's a difference between biology/chemistry and math, e.g. There isn't enough data out there on which to train MLs for the former.
I'm thinking here of LLMs as a lossy compression of all our published knowledge. And then agents doing some smart/heuristic search relative to that. They can surely find things that we have missed -- just because they'll tireless try all avenues of investigation that humans might have given up on -- but if the underlying data is not there they can't manufacture it.
LeCun has made this point elsewhere wrt world models and more ordinary physics. This issue is exaggerated in the case of biophysics and quantum mechanics.