They are poor at generalising from a small number of examples; this is why the real generalisation power is achieved in pre-training.