> The bad package version has also just disappeared from crates.io with no indication its been yanked.
So, yanked crate versions do have an indication on crates.io. (Here's an example: [1]) The Rust blog post uses the word "deleted", and I'm guessing a bit here, but I think they mean that literally, and that the version here is deleted, not yanked. And I think that would be more appropriate: a yanked crate is still downloadable by cargo, if your lockfile is locked to it already; yanking only prevents lockfiles from newly automatically acquiring a lock on that version. That wouldn't be desirable in the case of a compromised crate: you don't want a download occurring at all. The tradeoff of "break those with locks on the crate" tips to being worth it.
That said, I agree with you, though: I think this state (if it is "deleted" and not yanked) should be plainly indicated on the crate's versions page. (Even better would be if it came with a link to, e.g., the blog post or a RUSTSEC so that you could find out why.) (& I think perhaps the docs for yank should point out whether or not it is appropriate in the "compromised crate" scenario, and if not, what to do instead.)
> That wouldn't be desirable in the case of a compromised crate: you don't want a download occurring at all.
Sure you do; you just don’t want the crate to be made available to people trying to make use of the crate as regular downstream consumers. You still want the crate available for download for analysis. You especially want to be able to deterministically reproduce the vulnerable version of your software that you already built.
I would posit that most package ecosystems should treat “contains an exploit” as its own package state; mostly ignore them; but still be aware of their existence.
In other words, if there’s any other non-exploited option under your version constraints, the dep should resolve to the nearest non-exploited version, even if older. But if it’s the only version, or if you have pinned the exploited version, then the packager should see it, but normally refuse to interact with it—i.e. refuse to lock to it if it’s the only version; or refuse to fetch it if it’s the locked version.
I say “normally” because you should be able to bypass this with a flag / env-var to the packager that basically means “I’m building this for forensic analysis, not for running.”
Sure, I'm using "deleted" here to mean a third state more unvisible than "yanked"; I'm not trying to specify exact semantics about said third state, simply that it (a.) isn't yanked, and (b.) yanked's semantics are probably undesirable here and a third state has merit.
Yes, I agree with you that analysis is good.
I agree but think the problem is slightly more general. Consider that a package could be redacted not just due to exploit but also (among other things) for legal reasons, and that such reasons could be region specific. So a redacted package might or might not be available to manually download and some other metadata is desirable so that "audit" can notify you that you were compromised.
A package redacted for legal reasons should still be able to be built too! Presumably one or the other side of the relevant suit will need the package as evidence!
I would suggest that such packages should simply require authenticated access + manual “write an email to ask to have your account be put on a whitelist”-driven authorization. Such a package would still be there (in its redacted state); but attempts to download it would be a 401 (if unauthenticated) or 403 (if not on the whitelist) error response, which the packaging tooling would need to know how to handle.
(In any case, this “redacted” package state and my “exploited” package state could be two variants of the same logic, both following the same approach to locking/resolution, both requiring the flag to indicate that you understand what you’re doing and aren’t just going to run the resulting program. The only difference is that once you pass that flag, an “exploited” package would then just work, for anyone; while a “redacted” package would still have one more gate standing (server-side AAA) in the way.
I think maybe you're missing the significance of legally mandated redaction - continuing to provide access could well constitute a criminal offense depending on the specifics. And there may be many other reasons for redaction - as a sibling comment notes you certainly don't want to continue serving a package that contains sensitive private information.
I see the "exploit" case as a strict subset of redaction in which you do want to continue serving downloads to parties that explicitly acknowledge that they understand the classification of the artifact in question.
(Amusingly this exchange has led me to realize that as things currently stand a piece of malware can trivially be made illegal to distribute and thus extremely difficult to develop countermeasures for provided that the malware authors are willing to violate an additional law or three.)
> A package redacted for legal reasons should still be able to be built too! Presumably one or the other side of the relevant suit will need the package as evidence!
This doesn't make sense if the legal reason is accidental release of secrets or PI, in which case you absolutely do not want to leave it up.
I'm not talking about things like github tokens here, those can be revoked. But actual secrets or PI where that isn't possible.
I mean, unless the party you’re concerned about having your secret is the maintainers of the package store itself, what’s the functional difference between “it’s deleted from our servers” and “it’s marked private so only the author [or admins of the author’s org, insofar as user accounts are org-bound] can still download it”?
If I inadvertently committed (arbitrary example) my tax returns to a repo I'd hope that the operators of the package repository would promptly destroy all copies in their possession.
I think "deleted" is still always going to have to be a package state, realistically. A package could contain spam, abuse, copyright infringement and/or illegal content.
I don't mind malware being lumped in to that - the place to look up code for historical or analysis reasons ought to be version control, not crates.io
What about the place to look up packaged binary releases for historical or analysis reasons, esp. when the packaging system either builds such releases itself, or where it allows the submission of arbitrary build artifacts (where the build process that produced said artifact can be compromised in ways that have nothing to do with the source repo)?
> the place to look up code for historical or analysis reasons ought to be version control, not crates.io
The build infrastructure should make security incidents easier to respond to, not harder. I shouldn't be sent on a wild goose chase at a time when I'm potentially already dealing with a major incident. The tools need to "just work".