yes, it’s a very big loophole. and if it’s a generative model, you can just launder the data through synthetic generation/distillation to future models
I believe the big models currently get built from scratch (with random starting weights). That wasn't my point though. I meant a model created once, might be used for a very long time. Maybe they even release the weights at one point ("open source").
this is somewhat true but i'm not sure how load bearing it is. for one, i think it's going to be a while until 'we asked the model what bob said' is as admissible as the result of a database query
yes, it’s a very big loophole. and if it’s a generative model, you can just launder the data through synthetic generation/distillation to future models
Is that true? Do models get rebuilt from scratch each time or do they get iterated on?
I believe the big models currently get built from scratch (with random starting weights). That wasn't my point though. I meant a model created once, might be used for a very long time. Maybe they even release the weights at one point ("open source").
this is somewhat true but i'm not sure how load bearing it is. for one, i think it's going to be a while until 'we asked the model what bob said' is as admissible as the result of a database query