I'm going to double-down on my stubborn, unpopular opinion. This is my best attempt at explaining my criticism:
- Alien metabolites are a low-prior probability hypothesis. Dimethyl sulfide is a long-postulated biosignature with no natural source, so, it's low-prior
- The paper's model fits Webb data—a handful of photons—against no more than 20 candidate molecules, combined across all of their atmospheric models. Many of those gases are drawn from that low-prior "alien metabolite" class
- There's a much larger class of strongly infrared-absorbing gases, that can naturally occur in planetary atmospheres. Beyond those included in the 20 candidates. These (should!) have higher prior probability of occurring in Webb data than alien metabolites. (This class is so large and complicated, there's major spectral features in our own solar system's gas planets we haven't characterized yet)
- If you were to fit Webb data against that expanded class, those alternative hypotheses, you'd get a large number of 3-sigma detections by pure chance.
- The Webb data is too weak to distinguish between these. With only a few bits of information, you can distinguish between only a small set of alternative hypotheses
- This paper elevates the alien-metabolite hypothesis very highly, and that is why when it has a spurious statistical detection, it happens to be an alien metabolite detection. Because that hypothesis is overrepresented in their model
- The root problem is that since there's only a trickle of real data from this exoplanet, from Webb, it's unlikely one can infer anything super interesting from those few bits
False positives are acceptable if the goal is to generate leads to follow up on. If the detection was due to chance then it won't hold up to further measurement. There's few enough hits that we don't need to worry about being more rigorous (and potentially introducing false negatives) at an earlier stage.
Given the context, a publication seems appropriate. A high profile similar example is when neutrinos supposedly broke the light speed barrier. If the mass media misrepresents things that's hardly the fault of the scientists.