> LLMs are impossible to make safe in the same sense that humans cannot be made safe.
No.
LLMs are impossible to make safe in the same sense that a car designed as if it was the ~1940's would be impossible to make safe for its passengers during an at-speed collision. There's only so much you can do if you're committed to using plate glass, rigid steel everything, and leaving out occupant safety belts because they're unpopular and spoil the lines of the cabin. [0] Back in the day, "the people in the cabin are the crumple zone" was state of the art, but we've learned an awful lot about how to make much, much safer personal vehicles in the ~75 years since then. It'd be massively irresponsible to design and sell a car today that ignored the safety and engineering lessons we've learned since then.
"Funnily" enough, the major LLM providers have designed and are selling access to systems that they very much want to be used in situations where you need a reliable, safe tool... but they've -somehow- ignored one of the most fundamental lessons we've learned about the design of safe software systems that are intended to be used in the presence of attacker-controlled inputs. [1] What they've done is no less irresponsible than designing and selling a new car that conforms to the very latest safety regs of the 1940's... AFAIK, it's so irresponsible to design and sell such a car commercially that -in the US- it's a violation of federal law to do so.
As an aside: you may have seen this video already, but it's worth a look if you have not. [2] Though, the classic car in this crash is equipped with safety glass, so -sadly- you don't get to see all that fun.
[0] One of my great-grandfathers spent the remainder of his years intermittently using tweezers to remove shards of plate glass migrating out of his face that had been lodged in there during an automobile accident that he was fortunate enough to survive.
[1] For more on this, read: <https://news.ycombinator.com/item?id=48999644>
I want to raise the possibility that you (pixl97, simoncion) actually hold many of the same opinions but are clashing because of a different definitions the LLM / bad-thingy scope.
* Narrowly - The core algorithm that extends documents cannot be made safe, because it's a stochastic machine with no data/instruction separation possible. Unanticipated input can evoke arbitrary output.
* Broadly - The overall offering (centered on the document-extender algorithm) could be made safe by limiting its over-ambitious scope, treating the document-extender output as malicious-by-default, and sharply limiting what that output can drive or influence. Of course, that would exclude the berjillion-dollar stock valuation replace-all-humans stuff.
> ...but are using different boundaries for what constitutes "the LLM"
I'll make note that my original comment only used the term "LLM" in the phrases "the LLM companies" and "LLM-based systems". The latter use was in this footnote:
I acknowledge that my follow-on commentary -the one to which you replied- got sloppy with the terminology. I should have used the phrase "LLM-based systems", rather than "LLMs". I do feel that my original commentary was not at all sloppy with the terminology and made my position on the current state of the safety of the systems sold by the Big LLM Vendors and general understanding of where the bounds of the big pile of linear algebra and the bounds of the I/O to and from that pile lie clear.