Non-sentient technology has no concept of good or bad. We have no idea how to give it one. Even if we gave it one, we'd have no idea how to teach it to "choose good".
> In general we trust people that we bring onto our team not to betray us and to respect general rules and policies and practices that benefit everyone. An AI teammate should be no different.
That misses the point completely. How many of your coworkers fail phishing tests? It's not malicious, it's about being deceived.
But we do give humans responsibility to govern and manage critical things. We do give intrinsic trust to people. There are people at your company who have high level access and could do bad things, but they don't do it because they know better.
This article acts like we can never possibly give that sort of trust to AI because it's never really on our side or aligned with our goals. IMO that's a fools errand because you can never really completely secure something and ensure there are no possible exploits.
Honestly it doesn't really seem like AI to me if it can't learn this type of judgement. It doesn't seem like we should be barking up this tree if this is how we have to treat this new tool IMO. Seems too risky.
> they don't do it because they know better.
That's completely false. People get deceived all the time. We even have a word for it: social engineering.
> we can never possibly give that sort of trust to AI because it's never really on our side or aligned with our goals
Right now we can't! AI is currently the equivalent of a very smart child. Would you give production access to a child?
> you can never really completely secure something and ensure there are no possible exploits.
This applies to any system, not just AI.
> AI is currently the equivalent of a very smart child. Would you give production access to a child?
I mean this is my point! Why are we asking a child to do anything remotely important at all?
Maybe we should wait until the tech is an adult before we start having it do important things for us.
Mitigating the naiveness and recklessness of a child AI by attempting to lock down the environment as best we can seems foolish and short sighted to me and will probably not end well.
Whether it's being used inappropriately for production use and studying it to understand how to make it not be irresponsible to use in production are very separate things. What you're implying is that we should somehow magically leapfrog the current state of the art to a future version that solves all the problems with the current generation. Or, that we should ignore the technology entirely because developing it through the period where it's less robust than a mature human is too reckless.
The answer is that doing research isn't mutually exclusive with using the technology in appropriate ways. You can responsibly use AI while folks study threat models and model behavior for use cases that aren't able to be deployed responsibly.
> by attempting to lock down the environment as best we can
We literally do this as a best practice generally for traditional systems and human access. It even has a name: least privilege.