By reading the comments I wonder if perhaps OAI & ANT are not going to be “model” companies and they are going to be the ones innovating in many fronts and cashing in for the benefits of those advancements…
What I’m trying to say here is, what if, they are staying closed strategically so they keep their most advanced models close to their chest until they make an incredible discovery. Say they cure cancer. In this case, they are taking that IP collaborating with big pharma and cashing in on the upside of such discovery.
So, what if on top of being just another enterprise and consumer product that accelerates the way we all work and live, it also becomes this factory of breakthroughs and that’s where open source models can’t compete with them.
Just speculating…
We're already making huge advances toward curing cancer.
They're not coming from LLMs. They're coming from mRNA-based treatments and applying standard science to their basic principles.
And just to reinforce what djierardi said, you can't make successful new drugs and treatments for diseases just by thinking about it really hard. Practical tests are an absolute, unavoidable necessity—and not even just the final clinical tests; you have to be able to see how the molecules you're synthesizing interact with living tissue of various types, among other things, before you can even hope to test on humans.
Delegate that entirely to an LLM, and you'll end up giving clinical test subjects something that will melt their flesh, or coagulate their blood, or give them turbo-cancer. Because there is no comprehensive dataset that they can use to figure that stuff out purely by computation. We, as a civilization, simply do not yet understand biology and chemistry well enough to model it at that level.
So stop trying to figure out just how OpenAI and Anthropic are playing 12D chess and actually this situation is fantastic and will result in them saving the world. They're just greedy, selfish companies trying desperately not to be left standing when the music stops and all the chairs have disappeared.
I appreciate your points of view. I’m not trying to figure out the 12D chess these companies are playing nor I’m convinced these are the salvation of humanity. I just enjoy exploring all perspectives possible before arriving at a conclusion. It’s my defense mechanism against echo chambers.
You just made me think of LLMs as useful translators. I know they are not going to be the ones leading to the discoveries other ML mechanisms can. But LLMs are a great way to communicate though to a machine. Before it was only for those able to code, but now that problem is being resolved.
That is not to say that computer scientists are no longer useful, but it is now possible for everyone to talk to the computer and viceversa.
It sounds like you're positioning the LLM as "a computer that can speak human language (and also computer language because it's a computer)". If so, that's anthropomorphizing them to a concerning degree.
Even given the significant advances in coding models, everything I've seen described and experienced myself with them tells me that non-coders simply aren't going to be able to build and maintain nontrivial projects with them unless they get still better at it. By a lot.
And in the realm of medicine, the kinds of mistakes they absolutely will make, and defend, confidently, will get people killed.
Its plausible. Its just that the effort will be huge. LLMs can do a lot of stuff because they've ingested all of our published text. But, unless we're extremely lucky, there's just not enough data out there to solve this problem yet.
Maybe like with problems in maths: they've solved 700+ open conjectures, but how many did they try? Are the ones that they solved merely those that they put $10M effort into where humans just gave up because there wasn't any concrete benefit in putting in that sort of effort?
yes!
Sounds about right. About the only way you could justify the projection of making so much hand over fist.
But can they actually pull it off? Since the open models are kind of able to pull off the same things.
Maybe they can brute force faster, but not releasing allows another to scoop it.
Public closed-source models remain superior on paper, to the best of my knowledge.
Now, consider they have one, two or three generations of models being tested and toyed around before being released.
Those would be the ones used for breakthroughs.
What a movie. Act like religious whackjobs (A) who want to consume the world. In secret you have a bag of world-changing causes. The setback in the middle of the plot is the supercomputer answering something mundane to a serious question. Everyone feels defeated. But one underling is chatting with his pal subagent 3495. And he mentions the mundane answer. Then the subagent makes a critical connection; the answer is not mundane, it just needs a proper interpretation. Then they send the answer out (15 minutes later) cancer is cured. They use the patents to pay off the debts and then give them away to the public. The CEO moves to a shed on Más a Tierra with a talking parrot companion.
I don’t think the cure for cancer will be the result of the prompt “tell me the cure for cancer. Make no mistakes.” But instead applying real AI and ML supercharged by recent advancements to identify genetic mutations that could possibly cure cancer or other diseases.
Can I assume that you'll volunteer to be a human guinea pig to test these proposals?
We’re supposed to fluff up AI companies with cheap words here, not actions.
No thank you.
And I’m not proposing anything. I’m simply using my imagination here.
> Say they cure cancer.
A pet peeve of mine.
People keep mouthing this same fantasy. But cancer is so many different cancers. And "the cure(s)" aren't just 1000 different drugs, but maybe 10,000 different candidates for each that pass into lab tests, of which maybe 1% make it to phase 1 trials, and of those maybe 10% make it all the way through phase 3, each after 5-10 years of clinical trials and $100s of millions invested each. For the most part, AI-related biotechs are largely focussing on accelerating the discovery of good candidates. But that's a tiny fraction of the pipeline, which includes lots of lab work (which can be automated) and clinical trials (which OTOH require actual human guinea pigs and years of observation and assessment). How do you meaningfully accelerate that? Presumably by improving the odds of success in phase 3 trials from your initial candidates.
But nobody has proposed yet how LLMs can be a magic bullet here. At best they'll give you those 10,000 candidates per cancer type faster. But then you have all that lab work and clinical work to slog through, and all the likely failures you'll see in clinical trials. And there's just not enough data out there to train ML on to meaningfully improve the odds of success in clinical trials, since you'd really have to understand how your drugs work with all proteins and systems in every type of human body, e.g. "in silico" clinical trials. And we're only merely scratching the surface of that with stuff like OpenADMET.
That is, we can't even model well "how a human body works", much less "how those mechanisms go wrong in each type of cancer". But even if you could do all that, for the cures you'd need to figure out what novel, unusual, out-of-distribution thing (relative to your human model) you can throw into the system to avoid the bad behavior, that doesn't kill the person or interfere with other systems or get degraded too quickly or ... tons of other stuff.
Highly non-trivial. Maybe lower-hanging fruit is with a novel peptide or antibody, since those are better modeled with ML. But small drug-like molecules are galactic in number and not well modeled yet. ("Hey, what happens if I introduce this weird platinum-based drug into a body?" Well yeah, cisplatin is used for chemotherapy, but the body of knowledge needed to assess platinum-based drugs in a human body just doesn't exist, and ML isn't gonna intuit it from known data, nor "learn it" since that would require going down to QM, and then integrating that into its model of all human peptides and pathways.)
This is not a side-project for a chatbot company. Its a (multi-)generational project that requires buy-in from labs that can produce the requisite data.
(And okay so maybe I'm fast-and-loose above. Ideally a single "cure" would be to fix the apoptosis pathway, i.e. repair the built-in cell-death path for fast reproducing cells. But cancer cells mutate quickly and evade known avenues. Perhaps there's a generic fix? Plausible but same reasoning applies: ML doesn't have enough data to intuit a solution, if one exists. And there's no reason, a prior, to assume that one does exist. Only hope.)
Great take and useful for the thought experiment.
I just feel like your taking it all as a given/fact. And maybe you’re right, but I’m not that well versed into it to say things for sure.
What I'm trying to get at is that if we’re considering a black swan event, why not consider both the bull and the bear case or even the neutral one and at least accept that we don’t know which one it will be and it may well be a coin toss…
Perhaps using “cancer” deviated things into a specific subject, but it could be anything. A drug discovery, quantum computing, etc.
Again, I’m no quantum physicist or chemist, but as a world citizen, these feel within the realm of possibilities. Just like it destroying humanity or simply just being yet another technological advancement like the computer or the internet was.
—-
I guess the people working on computers could have argued there was a 10% chance it was going to end humanity and so the people working on the internet because AI was going to come about.
—
I’d say that my final conclusion is that we don’t really know and the signals we’re getting from founders, investors, the market, here, and other people we talk to is that NOBODY KNOWS. All we can do is to continue to speculate and make bets based on hunches.
Yeah. What I think is deceptive is something like solving Navier-Stokes with $10M of compute. The bigger question is how many conjectures and hypotheses they tried, and what fraction of those actually succeeded. If they expended say $1B on various open problems, and only 1% succeeded, is that the kind of investment that one will make? (The $10M alone is more than what would have been devoted to academics investigating these issues. But realistically one has to quantify the cost of the entire enterprise, and the probability of success vs failure.
From OAI we're hearing only about 700+ successes. Relative to what failure rate? The cost of success isn't merely the cost of each success, but rather the overall cost of all experiments, of which -- I'd guess -- a huge fraction failed.
I think that’s what VCs do. Only a fraction of investments succeed and the ones that do pay off big time.
Now, imagine having a mechanism that will constantly improve, so you can go from 1% success rate to 5% success rate in say a year.
I see your point. What I think could be different is the speed at which things can be done compared to conventional ways of discovery.
Think of a COVID/MRNA like solution for other areas.
I'm all in on that, and I own Moderna stock ... so there's that. But we sort of understand the pathway there: we want to introduce a protein that induces an immune system reaction. Traditionally you re-engineer a virus that produces that protein that produces the RNA that gets transcribed into that protein. But instead, with better tech, you introduce that RNA directly. Simpler and faster. And perhaps you can use that mechanism to introduce other proteins for other ends (as recently with its personalized melanoma treatments).
I'm all for the notion that ML can improve 1% to 5%. But I haven't seen anyone suggesting HOW that happens. So far, it looks like taking 1000s of candidates to 10,000 candidates, which just further clogs the pipeline. Using ML to automate lab tests seems like a win there. But still, pushing those through phase 1, phase 2, phase 3 clinical trials is the bottleneck. 90% of what enters that pipeline dies, and each involves many years and many $100s of millions of investment.
But as I noted: the quicker path would seem to be through peptides rather than small drug molecules. ML does pretty well with proteins/peptide chains and mRNA provides a way to introduce them (or injection, like GLP1's). Still requires all the testing, though, but presumably we know more about how they degrade, etc. over time.
It seems you just don't wanna see the bubble. Or maybe there isn't a bubble and that's your call to invest more? Amodei would be happy.
Oh, I see it. I’m just thinking 5–10 years down the line not short-term volatility.
Could there be more ChatGPT moments down the line in other areas like science, genetics, space and something we haven’t considered? I think there’s a chance.
I also think AI is here to stay and there may be some confirmation bias from the community regarding open source models. It’s like someone saying back then that Linux was going to take over… (?) Open source software seems to be served for technical people. Not for the average Joe looking for an answer or help generating a social media post or help writing an email.
Indeed, ML is here to stay. But there's a difference between biology/chemistry and math, e.g. There isn't enough data out there on which to train MLs for the former.
I'm thinking here of LLMs as a lossy compression of all our published knowledge. And then agents doing some smart/heuristic search relative to that. They can surely find things that we have missed -- just because they'll tireless try all avenues of investigation that humans might have given up on -- but if the underlying data is not there they can't manufacture it.
LeCun has made this point elsewhere wrt world models and more ordinary physics. This issue is exaggerated in the case of biophysics and quantum mechanics.
if they can actually do it, they're in big trouble because no one is going to let them hoard truly novel discoveries, eg: new physics, curing cancer.
Maybe they can’t keep the discoveries secret… idk.
But what they can do with them can make a difference. Especially with the access to compute and resources they have compared to anyone else. So, they still have a competitive advantage.
The one current example I can think of is Google. They have Alpha Fold and Quantum Computing. I believe they’ve seen advancements to the way they work and an improvement to the speed of progress. It’s just that it’s not what it’s talked about.
Yes. I doubt they’d be able to patent those discoveries without serious litigation from the healthcare sector. The models are all “polluted” with others’ IP.
Probability of "if"? I'll give you 1000:1 odds, if you're game.
IMO once AI can flywheel discovery, the entire international ip/patent system would simply break down - world is not going to allow US companies to automate innovation pipeline and horde / extract at scale, unless US somehow maintains full leading edge AI monopoly. Closed model discovering something a few months before open model and getting years of exclusivity rights is just not going to happen / last. If anything expect mass invalidation of IP which will disproportionately fuck over US incumbent IP leadership position.