> Low data regimes no longer exist in the age of LLMs, one can trivially generate a training and eval set and distill a good classifier on any domain within a day.

They do exist and you can't easily generate them. Distilling is also pointless.