Can you share more details? I ask because my experience suggests that models still require a decent amount of expertise to use for binary analysis (largely inferring because of use on other tasks of this level). I would expect models to always find "something" when you ask for stenographic techniques in the code, but with an extremely high false positive rate.
I don't think the diffs between Claude releases are that big. The amount of code in a diff doing sketchy stuff like looking into the host environment is going to be pretty small and obvious for the model. You can do things like ask for what an update included that wasn't mentioned in the release notes and stuff like that.