We see who signs up for Reality Defender and instantly notice traffic patterns and other abnormalities that allow us to see if an account is in violation of terms of service. Also, our free tier is capped at 50 free scans a month which will not allow for said attackers to discern any tangible learnings or tactics they can use to bypass our detection models.
How would you detect someone who tests a single image using a new free tier, and then (if successful) uses that image against a targeted customer account?
Working in a similar area (bot detection) I think it's very difficult to proactively stop such targeted attacks, but maybe in this space you can do something interesting like duplicate detection across a consortium.
We'd rather not tip our hand on any/all techniques used to discern actual users from bad actors and those seeking to reverse engineer, but suffice to say we do have are methods (and plenty of them).
You would need thousands to tens of thousands of images, not just 50 to produce an adversarial network that could use the API as a check.
If someone wanted to buy it, I'm sure reality defender has protection especially because you can predict adversarial guesses.
It would be trivial for them to build "this user is sending progressively more realistic, rapid responses" if they haven't built that already.
We see who signs up for Reality Defender and instantly notice traffic patterns and other abnormalities that allow us to see if an account is in violation of terms of service. Also, our free tier is capped at 50 free scans a month which will not allow for said attackers to discern any tangible learnings or tactics they can use to bypass our detection models.
Yes, of course new accounts should be scrutinized.
But what I find more interesting is how you prevent someone from training a model adversarially via one of your legitimate customers.
Wouldn't any of your customers that use your service to make a decision about something uploaded by a user be an attack vector?
How would you detect someone who tests a single image using a new free tier, and then (if successful) uses that image against a targeted customer account?
Working in a similar area (bot detection) I think it's very difficult to proactively stop such targeted attacks, but maybe in this space you can do something interesting like duplicate detection across a consortium.
We'd rather not tip our hand on any/all techniques used to discern actual users from bad actors and those seeking to reverse engineer, but suffice to say we do have are methods (and plenty of them).