TBH, I am waiting for Mathocalypse-level progress in biology and medicine, but I guess that's not happening as fast, because it's the real world, human lives, etc., mostly etc.
TBH, I am waiting for Mathocalypse-level progress in biology and medicine, but I guess that's not happening as fast, because it's the real world, human lives, etc., mostly etc.
Math is completely abstract and the base rules and mechanics of it are some of the most clear cut and formalised because that’s the entire point of the subject.
Biology is the complete opposite of that.
What if you could solve this bottom up, e.g.: Math -> Physics -> Chemistry -> Biology. I'm assuming that's how it would in theory play out.
Nonsense. This is like when people used to say "AI will never be able to write code as good as a human!" Biology will become a solved problem.
I don't think that's at all an accurate characterization of prior skepticism, and indeed it's quite obvious how the systems we've built to improve AI code generation work, and they were also an obvious thing to try (in parallel with the models themselves getting more capable).
Despite the improvements, though, we still see plenty of code that is "slop", because it's directed by someone with little idea of what to do, but also because it's harder to test for (or even measure) code quality than correctness. The "solved problem" rhetoric depends very strongly on your standards.
In other fields the problem is similar, and typically worse. Plenty of AI music sounds acceptable, not clearly flawed, and even mildly "interesting" to me; but I think this is much more a commentary on music than it is on AI. Prose continues to be abhorrent in nearly every application; while I've seen examples of people getting really impressive output, it takes a level of effort that defeats the purpose of almost everyone who wants AI to assist their writing (i.e., just generate a comprehensible report on some findings and get on with their day). Art is a whole different world; the quality of results clearly strongly depends on the quality of prompting, it's much better/worse in some genres than others, humans vary widely in their ability to notice "mistakes", etc.
Eventually maybe, but the problem is hard from both directions. Answering basic questions such as "what does aspirin actually do at a biochemical level?" or "design a drug to reduce beta amalyoid plaques" are both very non trivial. Nevermind something complex like 'cure cancer'.
it can be simulated to a very high fidelity.
There’s no simulation you can do today that can tell you that your drug will likely make it through phase 3 trials. The best we can do is simulate reactions of a few key molecules, we’re nowhere near cell or organism scale simulations.
Observing a faithful simulation is not at all the same thing as understanding what has been seen.
we do have high precision microscopes that can see molecules, so it is possible to build closed system with feedback loop that feeds back into AI at very small scales
It's often not necessarily about visibility, but real iteration time.
For instance, let's say we are doing biology on plants, which is still much easier than doing human trials. No matter how motivated claude might get, you aren't getting a corn plant to grow in 40 seconds. Real world research pipelines involve stages where even greenhouses don't do, and sacks of grain are shipped to Hawaii, or Brazil, or wherever, to try to get practical data for a given test.
If your computer simulation is so accurate you can skip a step now done with biology, great, but it gets harder the larger your problem is. I might see if a molecule kills bacteria kind of quickly, but what does a molecule do to a real human being? Not so easy.
I don't follow how "we can see tiny stuff" = "it is possible to build closed system"?
Seeing tiny stuff doesn't magically give you enough info for Claude to reason with the system predictably (the way it can with source code).
It can start to reason about what it seems under the microscope. You just need to train it on images and data from microscopes.
Have you looked at cells and tissues under a microscope? There’s patterns upon patterns inside patterns under and over and through other patterns.
There’s no well-labeled dataset for all this because we don’t know everything we are looking at.
Take a neuron. You can stain it so you can tell apart an axon and a dendrite. But if you don’t it’s very possible to miss or mislabel the axon, because unlike the textbook neuron, the axon isn’t always obviously oriented in a way that promotes ID.
And that’s just one example. With more complex samples, you also have a lot of edges which AI models don’t segment very well.
You largely explained a day in life of a SWE working in a complex codebase spanning (tens of) millions lines of code. In complex products we always deal with a great degree of uncertainty and due to the ambiguity and high complexity we usually have no simple way of proving that the thing (TM) we designed is going to be working as advertised in all cases. Because the problem space is in-exhaustive, and to increase the likeliness of the solution we designed, we run the experiments, and not on a sole machine but clusters at data-center scale. Even then you get "bug reports" and that's not because engineers were lazy but because the domain is genuinely complicated, and ever changing.
So, I'd say it's a pretty similar process, and yet we have the AI models now almost completely integrated into the SWE domain remediating significant parts of its complexity.
What is it with SWEs thinking every problem can be addressed in terms of SWE, while at the same time being unaware of how little they understand the problem they are trying to address?
Do you know what loading a sample into a scanning tunneling microscope involves?
the transition is in-silico => in-vitro => in-vivo
AI already can do in-silico simulation
add microscopes and tiny precision actuators, and we can build closed feedback loop in-vitro system
once we have in-vitro, it will be easier to go in-vivo
Why would number 2, which we’ve been able to do for decades, possible affect number 3 on your list?
No its not, this is complete dunning kruger, please provide an example.
Indeed, actually, one of the limitations preventing AI from making more contributions is humans. Human emotions, laws, morality, responsibility, and so on are all part of these limitations.
I've always thought openai could actually create an agent that can control various physical robotic arms and things like that, and then continuously iterate on solutions that can participate in the real physical environment. This could accelerate automatic iteration for many things that require human operations, such as complicated experiments, assembly adjustments, and testing.
Otherwise, AI is always trapped in virtual systems and still needs human intervention. But even though AI is already very powerful now, there is still a limit to how much humans can handle or assign tasks. Many times, AI has already finished processing something, but I don't even have time to assign the next deeper level of tasks.
People hope AI can improve productivity, but at the same time they're afraid of AI actually getting involved in reality, while the valuation of the entire AI industry is so high.
This kind of wanting to move forward but not moving forward makes humans especially uncomfortable.
Google does this research not sure if the others are doing it. Embodied reasoning
https://deepmind.google/models/gemini-robotics/
They should put AI in a car so it can drive itself or create robots that use world models...
I expect that AI use in the more axiomatic kinds of research will be well in advance of sciences that have a heavy, messy empirical foundation, like biology.
I might be wrong, but I expect robotics to come before Biology. At least robotics is much easier to verify, most humans can look at a robot and let you know if it is working correctly or not.
Interesting topic, considering how many humans are unable look at themselves and know if they're working correctly or not!
Improved robotics would improve biology. Fast cheap accurate flexible pipette-bot would help in many labs. Imagine if a postdoc could demonstrate and the arm could watch and repeat.
In what way is robotics a frontier research area? (What sub-fields are considered frontier?)
That plus organs on a chip could be scalable and generate interesting data
first math; now its first integral w.r.t. time: physics, then second integral: chemistry, then third: biology
finally, us
Keep waiting
Oh I'd hope we can do better. Cures with only theoretical evidence and an inscrutable explanation for why they work sounds terrible.
Specifically, it's because the verification loops are much longer in applied sciences. In biology you need to wait for wet lab experiments, clinical trials, regulatory steps, etc. before you can call something "solved".
Since sequential steps are slow, I think we're going to see massive increases in parallelization. I'm not sure how the regulatory frameworks will keep up honestly.
> we're going to see massive increases in parallelization
Why? This is not an AI problem, it's an industry one. And if it was simple, it would already exist.
Well, it matters more for AI because there are some limitations of human research that don't apply to AI. Not all of course, just mainly the fact that there's a finite number of great researchers. Other things are limitations that could theoretically be overcome -- for instance, it's "only" money stopping you from having more lab space as long as you have a suitable robot to run a lab.
> as long as you have a suitable robot to run a lab
Let me guess. You have never stepped in a lab.
> And if it was simple, it would already exist.
Not necessarily. Many things are simply gatekept behind the door.
Do you have any idea of the amount of money involved in pharmaceutical R&D?
Yes, I do. My comment was a general sentiment saying that sole inexistence of something is not a proof that that something has to be very complex. Pharma is no different when it comes to people relationships and/or fear of somebody else stealing your chunk of expertise and/or fear of some other company taking a significant part of your revenue. Gatekeeping is one form of "preventing" or slowing down the process, and we see it in various domains regardless of their complexity.
We see it happening in SWE as we speak - people from and outside the tech who had no or very little experience with programming are now starting to deliver their ideas into products without devs or very few of them. We also see this within the SWE where strong domain experts are now able to deliver ideas into code easier than ever, and by themselves, and for which they would previously need a handful of programming experts. We also see it in other way too, strong programming experts are now able to pick up other domain easier than ever.
This process is something that I believe it will (and hope so) inevitably be picked up by the pharma too so the myth about the complexity will become broken.
We already have as much parallelization as companies can afford: Hundreds, if not thousands of new corn varieties, coming from one lab. The budget is the limit. Testing is just expensive.
Process improvements come from failing faster, so you spend less money on your test. Finding the actual answer without testing would also be wonderful, but LLMs might be fast, but they aren't magic. Something like Alphafold saves time in some steps, but it's not just "hand me a candidate", but find completely novel verification mechanisms, which would be just as useful for humans. Either way, parallelization is unlikely to increase.
Amdahl's Law [1] is not your friend here. Even a thousand women can't produce a baby in one month. Trying a million things in parallel doesn't help much when the possibilities involve cosmic-level numbers and you need to iterate on past results to make progress.
[1] https://en.wikipedia.org/wiki/Amdahl%27s_law
> wet lab experiments, clinical trials, regulatory steps
I wonder what the unlock would be, if any, of having simulations substitute any (or all, in time) of these steps. Akin to the method used to train FSD cars, or robots.-
In my untrained opinion, I don't think that will be as much of a cheat as it might seem. Or more accurately, creating a simulation accurate enough to where it would work for this purpose would be where all the breakthroughs would have to be made. Building a perfect simulation requires an understanding of the system being simulated, that I don't think we fully have of molecular biology -- if we did, I suspect we'd already have many of the answers we are seeking when we do this kind of research.
But I'm no biologist so if someone else is, feel free to correct me.
You may recall Google published AlphaFold in the last few years -- its main goal is precisely this. Combining up that tech with these pipelines is likely to speed things up. It looks like things are already speeding up a lot based on the press release, though. This seems likely to be an area where we'll see some compounding returns over the next couple years.
Indeed, I do recall, now that you mention it.-
By the time we can simulate a human body to the degree that clinical trials aren’t necessary I think we have solved disease a long time ago.
"All models are wrong, but some are useful."
All valid points on being able to simulate well enough meaning we would have nearly solved everything to begin with.-
I wonder if, at some point, AI's will perform to such degree they are able to construct "black box" simulations as needed, that nobody understands BUT that are sufficient to empirically experiment on ...
... much in the same way LLM's function now: In an aggregate sense, they are bloack boxes.-