(3) seems like a problem in its own right? Basing science, traditional or newfangled ML, on such small amounts of data looks pretty weak.
(3) seems like a problem in its own right? Basing science, traditional or newfangled ML, on such small amounts of data looks pretty weak.
I worked on some of the very best funded plant research out there. When it comes down to it, there's enough variation caused by confounding factors, and it takes so long to capture more data, that almost everything anyone tries cannot be called a success or a failure for years, because the individual measurements for one small plot of land somewhere just don't mean anything. Once you do an entire experiment for a season, which takes months, and you grab the little noisy data you have, and turn it into real rows, we were down to very little.
You can do more tests on smaller things, like checking if some protein will kill some cells of a pest, but making sure a plant produces it enough that it actually does something significant to the real, live pests, that it's not toxic, and it doesn't harm the plant's yield massively (as it's now spending time producing your pesticide) is still going to take years. We might be able to fold proteins, but the kind of things we'd need to really simulate plant biology well enough to not need years of failures are still very far away.
And it's far worse in medicine, as with plants at least nobody has ethical concerns if they fail and die, and nobody needs to get consent from a corn seed. Getting to 50 actual data points from many medical studies is already a lot of effort. And imagine when it's a long term study, and you need to follow patients for 30 years, as theym move, or die, or decide to stop participating, or who knows what.
Ethical concerns might become an even bigger bottleneck in the future. It's sad what we're doing with millions of rodents each year.
In chemicals and materials, 50 rows of good data is a really solid study. That’s e.g. a 3x4x4 experimental design (assuming replicates for each condition get averaged into a single row). If you managed to prep that many samples correctly and obtain consistent characterization data across all properties of interest, you’ve easily got a paper. It’s also kind of malpractice to jam this type of data (few samples, wide rows) into modern ML models. There are plenty of simpler statistical methods that will tell you what’s going on, and even then a well-made plot might be good enough. The difficulty is not in drawing insight from the final numbers, it’s almost always in how those numbers came to be in the first place.
Thus the reticence of science-oriented companies to invest heavily in these mass data-gathering exercises to feed ML. It’s damn expensive, and almost always leads you back to raw data issues, not breakthrough discovery. Doing it without a set purpose in mind is even more likely to yield garbage.