Is there a path where compute gets poured into developing, and then eventually using, computational models that supersede the shortcoming of the existing computational approximations you mention, which thus makes computational analysis and developments possible at rapid scale?
In an area I am more familiar with though still not my domain: when the stiffness matrix and computational linear algebra methods collide, it unlocks a world of computational structural design and analysis by commoditizing the underlying mathematics. This is a simple example, but you get the idea.
Obviously with CFD, mechanics, etc there is always massive work being done to try to unlock new computational methods, faster/parallelizable computational methods, differentiable ones, etc etc, but the general feeling is that the core physics are well modeled (NS millennium problem not withstanding) and are not lacking in predictive power for useful applications. I would have assumed that the same would be generally true for computational biology etc, so just curious to hear you elaborate more. It seems plausible to me that turning the brute force of “the models” (as in, swarms of AI compute) to work on/develop/solve/address whatever representational problems there are in the computational approximations/numerical methods you mention, particularly if that is a bottleneck to performing more extensive methodological/experimental exploration in silico to enable better exploitation/use of limited resources in wet lab.
Again, I know nothing about computational biology, just stating my intuition and curious what your take is. Probably the limits of my knowledge can be demonstrated by the fact that the only modeling methods that come to mind as being vaguely relevant would be, like,… molecular dynamics, maybe discrete element method in a weird way but doubtful, and various pop science understandings of protein folding etc.
Computational biochemistry is what I'm currently pursuing, but I'm pretty new to the field so take everything I say with a grain of salt: Biochemistry is a really really really tricky problem, for a couple reasons:
1. Biology has a very nasty habit of not cleanly abstracting. It's not like we can treat every protein as a "diffuse mass" without losing a ton of important interactions. This is unlike something like CFD where you can treat a mass of water molecules with certain constants. Think of something like cancer: a small mutation to a DNA strand can cascade into a tumor. You have to deal with the mutation happening at an atomic level while scaling to a whole organism (yes I know there's models that can approximate this, but they can't capture all the interactions).
2. This brings us to modelling at the atomic level. To model a system of atoms with the Schrödinger equation, it naively scales O(6^n) per particle. Note that means every nucleus and electron, so if you have say 10 carbon atoms and 22 hydrogen you're already at 10^127 operations. We can cut corners a bit and end up with something like CCSD which scales O(n^7), or cut more corners and scale O(n^3) for DFT. DFT is pretty much the last method where bond forming and breaking is modelled, so after more corner cutting you lose losing reactions, which are important for understanding why something is happening. Most protein folding simulations (moreso prior to alphafold) run a very crude approximation that models atoms as balls on springs (things like AMBER and CHARMM), which obviously can't handle reactions.
3. If this wasn't bad enough, you also can only move through time at about 1 femtosecond (10^-15), so it takes a ton of simulation steps to get even a nanosecond in. Forget millisecond or second lengths where things start to happen.
So, while the math of the physics exists, computationally it's practically intractible. There's some exciting work with something called Machine Learning Interatomic Potentials (MLIP) which approximate DFT using a neural network which are about 1000x faster than DFT, and iirc scale much more linearly (look at things like MACE and orbmol if you're curious). So we're making progress, but we've gone from so ridiculously impossible to pretty much impossible for large systems.
I have a couple ideas about developing custom microchips to make this faster, which is why I'm pursuing this, but there's still a massive way to go!
To add to this I think there are “reductionist sciences” and “systems sciences”. Reductionist sciences are ones, like physics where there are lots of copies of a thing with near identical properties at different scales (the periodic table would be considered an incredibly short list and that seems to be one of the longest list of “different things” in physics).
Systems sciences are ones where each scale has lots of things with unique properties. If you think of “biological things between 2-4m” the amount of unique objects with unique properties and unique interactions is absolutely mind boggling. These sciences generally do not lend themselves to clean highly parsimonious models. The best you can do is coarse-grain a thing as a system (some very roughly defined set of interactions between different things) with certain behavior while accepting that you are losing enormous amounts of predictive power.
I suspect AI will remain worse at systems sciences for a long time because the “possible space” of models for a particular system is much larger than in the reductionist sciences and navigating it takes a human taste and instinct that LLMs don’t seem to have (yet).
One other thought, I think another aspect of biological sciences is just how much effort it takes to do research. Hypotheses are a dime a dozen, but actually verifying these is quite difficult. It takes a lot of patience, time, attention to details, and even then it often fails to replicate.
I don't think LLMs will just magically be able to connect all the dots, because "garbage in, garbage out" still applies. And there's just too many invisible intermediate states, there is literally just not enough information to fully constrain a model. Hence my interest in scaling computational chemistry to get these intermediates :)
Thank you for such a detailed explanation! I was wondering why we don't see MD everywhere and things like protein folding are all based on ML.
Is there anything like a getting-started for working with MACE or orbmol, a 'hello world'? Like maybe simulating Allolactose binding to lac1 ?
Heh. Unfortunately computer science is really the weird one out where open source is the norm. A lot of computational chemistry knowledge is mainly transferred in research labs and through college classes, and you have to pay for a lot of the tools. But hopefully I can at least point you in some useful directions. Here's a bit of an overview of the CC workflow and free options (note I haven't tried all the free options):
1. The first thing you need to do is start with some atomic structure. You can often find pre-existing pieces (through websites like pubchem), but you need to cobble them together. I use chemcraft and gaussview for this, but those are paid options. I've heard good things about the FOSS program Avogadro though.
2. Once you've built the structure of interest, you want to calculate properties of it. Note that we only infrequently simulate the atom's movement, since that's quite expensive. There's a lot of techniques that let us get the most bang for buck, like calculating transition states, or states of lowest energy which we call intermediates, or running something called an IRC, which follows the molecule to its lowest energy point (note we pretty much always think about these things in terms of energy). The traditional programs I use for this are Gaussian and ORCA (ORCA is free but you do have to create an account on their forum). Newer methods include MACE and orbmol, which are pretty much only used for trajectories (e.g. running through time) as far as I know. This newer crop of programs are much more FOSS, but they aren't as rigorous so my lab uses them only as a "rough draft". I pulled in one of my current notebooks if you're curious to try it out: https://github.com/smj-edison/qc-tools/blob/main/examples/ml...
3. Interpreting results/debugging problems. This is currently done in my workflow using a python script to extract values from the job runfiles (oh yeah, did I mention the terrible DX? A lot of this stuff is done by hand-writing a job text file and then using regex to parse the logfile). We do a lot of analysis based on relative energies (this is something called transition state theory of reactions).
I definitely glossed over a lot of details, but this is long enough as is. Definitely let me know if you have questions! If you do want a "hello world" though, I'd definitely check out the notebook I linked to. You'll have to create a file called "start.xyz" with some sort of chemistry program (or use AI to pull one for you), and then you can run the system through time. I would use a system I'm researching right now, but I'm not sure what our lab's policy is on broadcasting things to the world.
There's also so called linear scaling DFT