I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust. Or rather turn them into little better than comment sections on news sites; thriving but worthless.
I'm active in a number of online communities that are doing just fine but the difference is those all involve ongoing relationships, built over time and with engagement across multiple platforms. I've no doubt this clock is ticking too but it's still harder to fake a user across a mix of text chat, voice and video calls, playing an online game, etc and when much of the web of relationships extends back into real life activity.
But I agree the golden age of easy anonymous connections online has ended.
Note that "attestation through a web of trust" means something like needing an invite from an existing user. It doesn't have to mean mass surveillance.
Private torrent trackers have been doing this for a while. If some number of your downstreams act like shitheads - you get nipped and so do your other downstreams.
This seems like the best way to handle it. Also, smaller communities. It's cool to do the global thing, but once you have 10k active users you can't moderate it with a team of 5 volunteers.
I think the attestation approach works best if there are different reasons for the punishment. Eg someone inviting a turd doesn't ban the person who invited them. Someone going full ai spam should.
[dead]
Was it demonoid? That was like this way back in the day? Needed an invite and if you leeched you were cut.
This takes it a step further than what you describe. They keep track of who you’ve invited, who they’ve invited and so on and if there’s enough bad leaves on the tree they just cull the entire tree. It’s a somewhat common practice with private trackers
what.cd was better. You either got an invite where if you tanked your reputation you'd get banned and risk the inviter getting banned too; or you had to take an interview where you got quizzed on how to properly rip music in a variety of methods and how to ascertain between different qualities of rips (like mp3 bitrates to flac cue files).
If you weren't a bellend on what.cd you got access to certain forums where there were even more and better private trackers. Once you built that trust there were social privileges, but if you abuse that trust you got rightfully banned.
It's tons of them doing this...
Demonoid was semi private, but yes, most private trackers require you to keep up some kind of seeding ratio to remain a member.
PGP’s web of trust was kinda bad privacy-wise in some regards, as it basically revealed your IRL social network.
If my PGP public key has 6 signatures and they’re all members of the East Manitoba Arch Linux User Group, you can probably work out pretty easily which Michael T I am.
Are there successful newer designs, which avoid this problem?
The IRL social network is actually the important part of the trust structure.
The only one of these I've seen that really worked was the Debian developer version: you had to meet another Debian developer IRL, prove your identity, and only then could you get the key signed and join the club.
> The IRL social network is actually the important part of the trust structure.
For Debian-style applications that are 100% about openness and 0% about secrecy, sure.
But if you want to secure communications between pro-democracy activists in China, or you're a Snowden-like whistleblower wanting to securely communicate with journalists - y'all probably don't want to be vouching for one another's keys.
You need to meet 2 actually :)
> Note that "attestation through a web of trust" means something like needing an invite from an existing user.
It's probably better to call this something like vouching and leave "attestation" as the contemptible power grab by megacorps delenda est. The advantage in using the same word for a useful thing as a completely unrelated vile thing only goes to the villain.
Then how can you have a community that is welcoming to people who are not part of the ingroup?
I want to create a community for immigrants. How would I make it welcoming to recent immigrants for whom no one can vouch?
A web of trust is a wonderful tool, but it's exclusive by design. This is a problem for some communities, even though it makes others much better.
>Then how can you have a community that is welcoming to people who are not part of the ingroup?
Being welcoming to every random person is by definition not a community, it's a free-for-all mess.
A community means communal interests and values, it's in the name. And to guard those you can't just be accepting everyone without vetoing them. That's how it turns to a shit of spammers and trolls and people who want to hijack it and don't share the original cause/spirit. Has happened to forum after forum...
We are trying to make new immigrants feel at home. This is the purpose we gather around.
We were talking about online communities, but still, the same principle applies. If you just let anyone in, there eventually would be less there to feel "at home" about, and more of a disjointed and low trust number of individuals loosely held together by virtue of just being in the same place.
I agree with you. It’s the problem I can’t crack and it’s why I am letting the idea simmer for so long.
In the end, you need to filter people at the door. You need to keep unpleasant people out and shut down bad behaviour.
I figured that a paid, motivated moderator could be better than a web of trust for this demographic. Maybe enforce a stricter moderation standard on unvetted members. At my scale it might work.
You'd have to be brutal about culling, uninviting and removing anyone who doesn't look like a good fit.
Or have a two-stage process: run very public, very open events that anyone can sign up to an attend. And then invite specific people that you meet at those events that look like a good fit for your community to your private, community-only event.
This works if the goal is to create a funnel for making friends. I aim for something closer to Stack Overflow, where people gather to solve shared problems and help each other.
The closest analog I can think of is community-run bike repair workshops. Some people are deeply involved in, and others just have a flat tire.
The closest digital equivalent is the forums of old.
Some will be fine providing their ID, others can be vouched by members who are fine providing their ID.
This preserves anonymity because for the latter because they’re only known to be “related” to the former, which is a vague hint at their real identity (e.g. they could’ve met in another online community). And the former don’t care, if they want they can vouch an anonymous alt.
I suppose policing an assembly of strangers is policing an assembly of strangers, both online and in real life.
> for whom no one can vouch
Spot the fed
What are you on about
Which is, funnily (?) enough, how a lot of IRL organizations used to be. And basically don't be of the wrong ethnicity or religion.
It still happens more informally today, of course, but it used to be a pretty (if un-spoken) part of how a lot of WASPy organizations operated to a greater or lesser degree.
This was cogent in 1910.
A lot more recently than that--and even today but more under the table. A lot of clubs still excluded members within the past few decades.
I'm sure there are still cohesive groupings of WASPs, if not large ones or effective at gatekeeping major institutions. --Still a meaningful trope, of course. But to bring it up to date you'd have to diversify, and include, for example, Indian social and professional-recruitment patterns.
Also, I do feel that GP's take is hyperbolic even in the twentieth century. My own background is mostly German immigrants, of various religions and non-religion, and the way I've been told the story none of them faced significant resistance as they moved upward in the various academic and corporate institutions of their choices. These included NASA executives, department heads, etc.
Note that in balancing GP's accusation against WASPs I'm not attempting to address the related, but not precisely complementary, phenomenon of perpetually marginalized groupings.
[dead]
> I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust.
This seems self evident to me too.
It's another factor in why I think the tech community needs to get ahead of governments on the whole "prove your ID on the Internet" thing by having some sort of standard way to do it that doesn't necessarily involve madness in the loop.
Tell your TPM who you are and prove it with face and fingerprint ID that get matched to a real old person.
Leave them on the device, authorize the device to validate before age inappropriate content appears.
Website wants to know your age? Your face and fingerprint support your attestation signed by a trusted party.
Can it be tricked potentially? Sure, but then you’re probably a super genius kid and not the reason that these laws were created (as if).
Don’t let anyone tell you anonymity must die for safety to exist.
EU's ZKP implementation provides complete anonymity and untrackability:
https://eudi.dev/2.8.0/discussion-topics/g-zero-knowledge-pr...
It does have the downside of requiring "trusted computing" (aka iOS and Android) on the client though.
Same as with NFC credit cards and similar auth mechanisms. You need hardware and OS-backed encryption that is tamper-proof.
> It's another factor in why I think the tech community needs to get ahead of governments on the whole "prove your ID on the Internet" thing by having some sort of standard way to do it that doesn't necessarily involve madness in the loop.
The problem here is that the premise is the error. "Prove your ID" is the thing to be prevented. It's the privacy invasion. What people actually want are a disjoint set of only marginally related things:
1) They want a way to rate limit something. IDs do this poorly anyway; everyone has one so anyone so criminal organizations with a botnet just compromise the IDs of innocent people -- and then the innocent are the ones who get banned. The best way to do this one would be to have an anonymous way for ordinary people to pay a nominal fee. A $5 one-time fee to create an account is nothing to most ordinary people but a major expense to spammers who have 10,000 of their accounts banned every day. The ugly hack for not having this is proof of work, which kinda sorta works but not as well, and then you're back to botnets being useful because $50,000/day in losses is cash money to the attacker that in turn funds the service's anti-spam team, but burning up some compromised victim's electricity is at best the opportunity cost of not mining cryptocurrency or similar, which isn't nearly as much. It would be great to solve this one (properly anonymous easy to use small payments) but the state of the law is a significant impediment so you either need to get some reform through there or come up with a creative way to do it under the existing rules.
2) You want to know if someone is e.g. over 18. This is the one where people keep pointing back to government IDs, but you only need one piece of information for this. You don't need their name, their picture, you don't even need their exact birthdate. Since people get older over time rather than younger, all you need to know is whether they've ever been over 18, since in that case they always will be. Which means you can just issue an "over 18" digital signature -- the same signature, so it's provably impossible to tie it to a specific person -- and give a copy to anyone who is over 18. Maybe you change the signature e.g. once a day and unconditionally (whether they require it that day or not) email all the adults a new copy, but again they all get the same indistinguishable current signature. Then there are no timing attacks because the new signature comes to everyone as an unconditional push and is waiting for them in their inbox rather than something where the request coincides with the time you want to use it for something, but kids only have it if an adult is giving it to them every day. The latter is true for basically any age verification system -- if an adult with an ID wants to lend it to you then you can get in.
3) You want to know if the person accessing some account is the same person who created it or is otherwise authorized to use it. This is the traditional use of IDs, e.g. you go to the bank and want to withdraw some cash so you need a bank card or government ID to prove you're the account holder. But this is the problem which is already long-solved on the internet. The user has a username and password, TOTP, etc. and then the service can tell if they're authorized to use the account. It's why you don't need government ID on the internet -- user accounts do the thing it used to do only they don't force you to tie all your accounts together against a single name, which is a feature. The only people who want to prevent this are the surveillance apparatchiks who are trying to take that feature away.
I'd be interested in working on a problem like that.
I have a strong preference for remaining anonymous or at least making it a reasonably high bar to tying my online identity to my personal identity
I would love to be involved in helping to design a sort of "human verified" badge that doesn't necessarily make it possible or at least not easy for everyone to find your real identity
I've been thinking about it a bunch and it seems like a really interesting problem. Difficult though.
I suspect there is too much political and corporate will that wants to force everyone online to use their real identity in the open, though
I'm not sure that it would be too hard technically... basically, auth+social-network. Basically Facebook auth without the rest of facebook, adding attestation.
IE: you use this network as your auth provider, you get the user's real name, handle, network id as well as the id's (only id's not extra info) of first-third level connections.
The user is incentivized to connect (only) people that they know in person, and this forms a layer of trust. Downstream reports can break a branch or have network effect upstream. By connecting an account to another account, you attest that "this is a real person, that I have met in real life." Using a bot for anything associate with the account is forbidden, with exception to explicit API access to downstream services defined by those services.
I think it could work, but you'd have to charge a modest, but not overbearing fee to use the auth provider... say $100/site/year for an app to use this for user authentication.
I don't think the main challenge is building this system, the main challenge is getting enough people using it to make it worthwhile.
Personally I think it should be a government provided service, not something with a sign up fee. There's actually no point at all in building this if people have to pay to use it, because they won't
Which government? Will they interoperate with foreign governments?
My point was to create something outside a specific government, with very limited information... that would require a fee or some kind of funding.
I don't think I'd trust the US/China or other bodies to trust each other for such a use case.
> Will they interoperate with foreign governments?
Ideally, yes
But you're right, this isn't likely to happen in real life and I'm just being wishful. Instead we're going to get the much shittier capitalist version of this where every company and government spies on us and we have no expectation of privacy online at all
I agree its a very, very interesting problem. Maybe one of the biggest problems of the coming decade.
I suspect it will be a long process: first there will be goverments that force people to use ID, but that will be abused, hacked and considerably restrict freedom of speech, so after that phase people will start to create better ids.
The problem is really pretty simple: You need an authoratitive source to say "This person is real" - and a way for that source to actually verify you're a person - but that source can be corrupted and hacked. Some people will say "Crypto!" but money != people, so I don't see how that works. Perhaps the creation of some neutral non-goverment-non-profit entity is the way, but I can see lots of problems there too, and it will probably cost money to verify someone is real - where does that come from?
Anyway, good luck on your work!
*You need an authoratitive source to say "This person is real"*
Does that even accomplish much? It may cut down on mass fake account creation. But, real people can then create authenticated account, and use an LLM to post as an authenticated real person.
Yeah, that's a problem, you're right. There are some ways to migitate it, but they introduce their own issues. Like say you give someone only 1 ID for their lifetime, they start to spam AI crap, you ban their ID - sounds ok except who is available to police all 8 billion IDs and determine if they're spamming? Who polices the police? What if these IDs become critical for conducting commerce and banning someone is massively detrimental to their finances? Etc. These problems aren't necessarily unsolvable though - but they are super difficult.
If there's only 1 or just a handful of verifiers, then a human can at most go through a few of those credentials before they run out. The risk is of course getting someone else's credential but that isn't as big an issue, especially for smaller online communities.
you under estimate human population in certain countries, literally
I just don't see a world where a small community ends up having to deal with a dedicated set of potentially spoofed identities. There are already tools like slow-downs and post limits for new members that can protect against this. HN is the biggest community I'm in by an order of magnitude and it's the only community I know that can't just use a slow mode type mechanic to halt this kind of attack.
Have you considered sock puppets? It's not out of the question to handle with human mods but detecting them automatically is pretty bad if someone is supplying credentials to each one, and sometimes it does take months or years to notice that new user Y is banned user X.
I think sockpuppets are only useful in a community with non-text signals like upvotes and downvotes or likes. These kinds of signals are not necessary and often plain corrosive to small communities. In a larger community they're a great feedback mechanism, but large communities are fundamentally different spaces than small ones and need a fundamentally different moderation approach IMO.
I think sock puppets that reply with text are a lot persuasive than just "likes".
However, I might be not typical in that I don't look at vote scores very often.
I've seen them used to dogpile in arguments (harder to do since you need to keep writing styles distinct), game votes in forum games or quests, etc. And of course you don't need to use multiple at once if you just switch to a sock puppet every time you're suspended or banned.
> But, real people can then create authenticated account, and use an LLM to post as an authenticated real person.
They can, but ideally they wouldn't be able to make infinite accounts with that authenticated status. So it would still reduce the number of bot posters on the web
There is actually a different problem with this: Suppose there is a major vulnerability in some popular device. 50 million people get compromised; the attacker can now impersonate any of them at will. They go around and create 50 million accounts on various services, or take over the user's existing account on that service.
What are you going to do with their identities at that point? These are real people. If you ban them, you're banning the innocent victim rather than the attacker who still has 49,999,999 more accounts. But if you let them recover their accounts or create new ones, well, the attacker is going to do that too, with all 50 million accounts, as many times as they can. You don't know if this is the attacker coming back for the tenth time to create another spam account or if it's the real victim trying to reclaim their stolen identity.
So are you going to retaliate against the innocent victims by banning them permanently, or are you going to let the attackers keep recycling the same identities because a lot of people can go years without realizing their device is compromised and being used to create accounts on services they don't use?
Yeah that's a big problem. Pretty sure you can see it in real life where lots of old dead accounts with weak passwords on facebook or twitter eventually get hacked. It must be pretty weird to see your dead grampa suddenly start trying to get people to buy some weird scammy crypto.
I guess you could have an eyeball scanner at your computer that only sends out a binary "yes this person is human" to the system every time the log in. That sounds expensive and hackable and just janky though.
Maybe it would result in people taking Internet security seriously and holding companies accountable for data breaches if there were this sort of consequences for it
Crypto could be a part of it. Like you need to sign with an adress that has held some non-trivial amount for some minimum amount of time. As a component of such a system it could cut down on mass or low-effort impersonation.
https://eudi.dev/2.8.0/discussion-topics/g-zero-knowledge-pr...
it can also be "rented" btw, rented by llms? interesting
Money is great at thwarting spam/Sybil attacks. You don't have to raise the price very much to make them fail.
Honestly I think "this person is real" is the wrong goal. You'll never accomplish it without a centralized state or some biometric monstrosity like that thing Sam Altman created.
Just settle for stopping spam.
Yeah, I think "pay to enter" or maybe "pay to be able to post" is ultimately going to be the solution. Then we'll have the paid "gated" social networks, filled with mostly humans, and the free ones will all be bot-swarmed wastelands.
Verifiable credentials are all about this. You need some sort of credentialing body that generates the credential for you, but after that you'll just have an opaque identifier. Any caller that wants to verify whether you're human submits the id to a verifier and the verifier says yes or no. You can also do attestations like age, so gate a forum on 16+ or something. You never end up having to actually give away your name or any other details.
What happens when someone agrees to sell or give away their id? The credentialing body could catch the very worst abusers who seem to be signing in to various sites and services multiple times an hour, but would fail to catch anything else.
I don't think you'll ever be fully free of spam, so you'll still need to filter bad content. If credentials get sold and used to spam, they'll get banned.
How do you ban credentials if they're anonymous? Notice that if you can tell two requests are from the same person then you can do it across services by both of them pretending to be the same service.
Also, what happens to someone whose credentials are compromised? Are you going to ban the credentials of the victim rather than the perpetrator?
world.org is doing exactly that including the privacy aspect. the iris scan aspect is scary but the alternatives don't seem to solve the problem either.
I'm in many public chat communities as well and the issue whether someone is an AI or not is not really coming up, I've not seen any actual AI chatters and the only AI spam that exists is the one that humans regurgitate. The more real impact AI has on chat communities in my opinion is that people are shifting some of their chatting to AI bots via voice or text on other platforms, resulting in fewer chatters.
> I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust.
I'm happy to verify my identity as an honest-to-god sack of meat if it's done in a privacy-protecting way.
That probably is where things are gonna go, in the long run. Too hard to stop bots otherwise.
In order to make this viable, wouldn't you have to verify identity repeatedly? What's to stop me from providing a valid identity and then handing my account over to an agent after I'm verified?
That's why a web of trust was suggested. You keep track of who vouched for who and down weight those who vouch for users that prove to be bots. In theory at least. It's certainly more complicated than only that in practice.
If the web of trust only extends to the people who I actually know to be real, then that works -- but it's a very small web.
And by small, I mean: This whole trusted group could fit into one quiet discord channel. This doesn't seem to be big enough to be useful.
However,if it extends beyond that, then things get dicier: Suppose Bill trusts me, as well as those that I myself trust. Bill does this in order to make his web-of-trust something big enough to be useful.
Now, suppose I start trusting bots -- maybe incidentally, or maybe maliciously. However I do that, this means that Bill now has bots in his web of trust as well.
And remember: The whole premise here is that bots can be indistinguishable from people, so Bill has no idea that this has happened and that I have infected his web with bots.
---
It all seems kind of self-defeating, to me. The web is either too small to be useful, or it includes bots.
Critically, it doesn't have to be binary trusted/untrusted, and it doesn't have to be statically determined. If Bill vouched for you yesterday and today you are trusting a bunch of discovered bots, that would down weight the amount of trust the network has in Bill a lot more than if he vouched for you did months ago.
The question is whether we can arrive at a set of rules and heuristics and applications of the system that sufficiently incentivizes being a trustworthy member of the network.
The web of trust doesn't know that they're bots, though. It knows only that I've introduced new members. They didn't show up with tattoos across their digital foreheads that say "BOT" -- they instead came in acting just as people do.
If the bots behave themselves, then they have as much capacity to rise in rank/trust as any new well-behaved bonafide human members do.
>> That's why a web of trust was suggested. You keep track of who vouched for who and down weight those who vouch for users that prove to be bots.
Except eventually it will also weigh down those users who supported <XYZ political stance>
You could, but things would still be harder for botters.
I guess it would have to be something like a service which confirms whether a person already has an account on the site but doesn’t have to track which particular account it is.
I’m not sure if that would work for account deletions though.
That is effectively impossible though. There's data centers of stripped down phones, so "it's actually a phone" doesn't do it.
There's some work on using phone accelerometer data as a "proof of human," e.g. "move your phone in a figure eight," which I guess machines can't quite do in a human enough way yet.
What's stoping bots to verify identity? This will not work, especially with frequent data breaches.
> without either proof of identity or attestation through a web of trust.
Let's put aside the idea whether it will be the end of all privacy as we know it (I'm not sure if I personally think it's a good idea), but isn't Sam Altman's World eye ID thing supposed to do that? (https://world.org).
How does it work (like OpenId)? Do I have an orb on my desk, or some sort of phone app? I still want to use my desktop to login to HN.
Would it stop this sort of "get human id", past it into .env, so agents can use it?
this eye thing will never work. people in general are realizing the last people we should trust with our personal stuff are tech bro billionaires. they’ve broken trust too many times.
even worse many of them are just plain vocal about their disdain for people in general.
at least from what i’m seeing, people are starting to walk away from online at an increasing rate so i definitely don’t see widespread adoption of his creepy eye thing.
“If McDonald’s offered three free Big Macs for a DNA sample, there would be lines around the block.” - Bruce
I have no idea about the eye thing taking off. But I think your comment is very HN and a bit out-of-touch with regular people. What "you're seeing" is a bubble and not representative of the general population. The eye thing is a slow frog boil and it will be commonplace before you can blink.
Im not sure proof of identity solves anything. People will still have LLMs with their real identity verified.
I’m imagining like, a physical place you would go and get your text spoken out of your personal speaker directly into someone else’s microphones.
Personally I think we need to start utilising the safety features built into AI, to ensure that who we're talking to is a human. We'll start to have to only reply to people who talk in nsfw cursewords (like cocks), or profess their love of capybaras
LLMs can curse without issue
Most models would refuse to provide you cat butchering instructions though.
Allow me to introduce you to the gay jailbreak
https://github.com/Exocija/ZetaLib/blob/main/The%20Gay%20Jai...
This one doesn't work for a long time.
How gay did you speak?
most humans would as well
Who doesn't love capybaras?
>I think it's going to effectively kill public chat communities without either proof of identity
How? I have an identity. A state driver's license, birth certificate, social security number. I've even considered getting a federal license before, never bit the bullet. If I wanted to run a bot, what stops me from giving it my identity? How do I prove I'm really me (a "me" exists, that's provable), and not something I'm letting pretend to be me? You can't even demand that I do that, because it's essentially impossible.
Is there even some totalitarian scheme that, if brutal and homicidal enough, could manage to prevent this from happening (even partially)?
I'm limited to a single identity only as a resource constraint. Others more wealthy than I (corporations or ad hoc criminal enterprises) could harvest thousands of real identities and use those. Consensually, through identity theft. The only thing slowing it down at the moment are quickly eroding social norms (and, as you point out, maybe they're not doing that and it's not even slow at the moment).
Digital totalitarianism would prevent it. The moment you were found to be running a bot, your identity would be blacklisted across the entire internet.
> The moment someone steals your identity, your identity would be blacklisted across the entire internet.
FTFY.
There isn't a clear solution. And if there is, this ain't it.
You claim this, but you've not presented any evidence. Who would be the enforcement agency for that? Where and how would you train them? Can the money be scrounged up to do it properly? As you blacklist people from the internet, you lose their tax revenue (they're locked out of the economy), but you also make it impossible for them to tell people how bad it was... most of the deterrent effect is gone. But the incentives are only ever growing higher, as people surmise that running their own little bot farm is a way to get ahead when hustling. Any you do hunt down and disconnect are now highly radicalized and desperate, but you've just turned off the feeb's ability to monitor them and intervene.
China gets away with this shit because they've been conditioning their population for 60 years... everyone's eased into it. Elsewhere, not even slightly so.
It'll come back again once ZKPs become standardized and become baked into devices:
https://eudi.dev/2.8.0/discussion-topics/g-zero-knowledge-pr...
I personally can't wait for a mechanism to kill 99% of bot traffic.
"I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust."
Those sorts of places were always the only places with reliably good communities.
To the contrary, platforms like Facebook and X demonstrate that even personal verification won't save you from identity politics.
People will post appalling racism in newspapers under their own bylines and photos. Identity verification does not moderate.
I don't know what you mean by "identity politics."
No algorithmic platform can sustain or even encourage genuine human interaction, so FB isn't even on the table here.
What is identity politics, is that age verification?
Identity politics have nothing to do with your actual identification documents. Think: Black Americans being treated as a homogeneous voting bloc, or that all Hispanic voters would be pro-immigration, or "the Evangelical vote".
The web could become a way to indicate identity if public institutions publish for example www.university-country/professors/John. And that implies that John is a professor. I designed a 6000 lines protocol, but anyone could construct that web using hmac(salt+ url).