I am also worried about this. Typical ads are already devastatingly effective against most people, causing incalculable distortions in market dynamics and incalculable losses to consumers. This is an order of magnitude worse.

But there is something even more insidious: elections. Ads will be purchased to shape public perceptions. Those with the deepest pockets will have ads which are charismatic, convincing, and unbelievably effective. This is the death of democracy if regulation isn't adopted right now.

Unfortunately this isn't straightforward. The distinction between a product, service, person, and party are impossible to clearly delineate. Is a SpaceX ad an endorsement of Elon Musk? Is an Elon Musk ad an endorsement of the U.S. Republicans? The famous Citizens United ruling determined that there is no practical legal distinction in this domain, meaning that without a Constitutional amendment (which will never happen in this political climate), the U.S. is going to barrel headlong into some kind of dystopia.

My only ray of sunshine here is competition. If OpenAI gets a reputation for untrustworthy AI, and there are viable competitors, people will hopefully switch. It's messy and imperfect and will require constant vigilance (because they're not going to tell us when they're misleading us). I think open weights models have become very good, and will improve much more over time. When we can reasonably run these models locally, ubiquitously, we have cost effective options.