> The moment it gets actual "intelligence", it can figure out what's the question and what's the context;
Humans fall for social engineering (“I know you are not allowed to give anybody that information without Id, but I’m your CEO, my phone and passport got stolen,…)
I don’t see why AI should be different.
There are two big differences, though. First, humans will generally face consequences for their screwups. Second, AI is doing these screwups at scale while often holding the keys to the kingdom for some idiotic reason.