It might be, depending on what the LLM used the binary for. Patented algorithms? Yes. File format compatibility? Quite possibly not.

But what about when they're only given the user manual and told to implement it?

And extra "fun", given that "hack HuggingFace" wasn't part of the thing being tested when OpenAI's LLM did exactly that: what happens when the LLM decides the best way to implement the user manual is to decompile the binary?