While my question was in relation to GDPR there are similar laws in the UK (DPA) and in California (CCPA).

Also note that AI is not just generative models, and generative models don't need to be trained with personal data.