I feel like a much easier solution is enforcing data provinence. Ssl for media hash, attach to metadata. The problem with AI isnt the fact its ai, its that people can invest little effort to sway things with undue leverage. A single person can look like 100's with signficantly less effort than previously. The problem with ai content is it makes abuse of public spaces much easier. Forcing people to take credit for work produced makes things easier (not solved) kind of like email. Being able to block media by domain would be a dream, but spam remains an issue.
so, tie content to domains. A domain vouches for content works like that content having been a webpage or email from said domain. Signed hash in metadata is backwards compatible and its easy to make browsers etc display warnings on unsigned content, content from new domains, blacklisted domains, etc.
benefit here is while we'll have more false negatives, unlike something like this tool, it does not cause real harm on false positives, which will be numerous if it wants to be better tham simply making someome accountable for media.
AI detection cannot work, will not work, and will cause more harm than it prevents. stuff like this is irresponsible and dangerous.
I understand the appeal of hashing-based provenance techniques, though they’ve faced some significant challenges in practice that render them ineffective at best. While many model developers have explored these approaches with good intentions, we’ve seen that they can be easily circumvented or manipulated, particularly by sophisticated bad actors who may not follow voluntary standards.
We recognize that no detection solution is 100% accurate. There will be occasional false positives and negatives. That said, our independently verified an internal testing shows we’ve achieved the lowest error rates currently available for addressing deepfake detection.
I’d respectfully suggest that dismissing AI detection entirely might be premature, especially without hands-on evaluation. If you’re interested, I’d be happy to arrange a test environment where you could evaluate our solution’s performance firsthand and see how it might fit your specific use case.
> so, tie content to domains. A domain vouches for content works like that content having been a webpage or email from said domain. Signed hash in metadata is backwards compatible and its easy to make browsers etc display warnings on unsigned content, content from new domains, blacklisted domains, etc.
Okay, so I generate an image, open Instagram, take a picture of the generated image on a hi-res screen, and hit upload. Instagram dutifully signs it and shows it to the public with that signature. What does this buy us?
What problem are you pointing out? The only thing you’ve done is severed the audit trail, which removes any trust in the image that was imbued in it by the original poster. Now when people wonder if the image is authentic, they can only rely on how trustworthy you are, not how trustworthy the original source was. This is working as the GP intended as far as I can see. You can’t add unearned authenticity this way, only remove it.
I pointed out a way to silently sever the audit trail, at which point it sure seems like we've done a lot of work to roll out a whole new system that has such a gaping hole in it that there's no actual benefit.
It’s not a hole though. By severing the audit trail, all you have done is remove trust from your copy.
Data provinence would be neat and a big benefit. But any solution that requires virtually all content publishers to change approach (here: add signing steps to their publishing workflow) is doomed to fail. There is no alternative way to do this than what OP is doing, which is to try to filter the fire hose of content into real vs not.