I get your point that plastics are relatively inert and may not cause noticeable harm (depending on quantity?), but I think it'd be wise to be cautious. See for example https://en.wikipedia.org/wiki/Plastic#Bisphenol_A_(BPA) .

I'd also consider plastic, and their additives, to be a lot bigger and longer lasting unknown than GMOs.