> For a federated system they sure talk a lot about their struggles around scaling centralized infrastructure. ATProto seems like it's entirely the wrong protocol for this sort of application.
Huh that's surprising to hear? What makes you think that? It feels like the whole point of this release is to make it easier to allow people to self host or federate whatever parts they want? That's what I got from their intro/opener!:
> But the centralised forge is crumbling. Relying on someone else for uptime, for features, and to be a good steward of your code hasn’t played out well. What has stood the test of time is email: Git to move the code, and email — an open protocol — to talk about it. At the cost of user experience, however.
> But the centralised forge is crumbling. Relying on someone else for uptime, for features, and to be a good steward of your code hasn’t played out well. What has stood the test of time is email: Git to move the code, and email — an open protocol — to talk about it. At the cost of user experience, however.
> Tangled aims to give you the best of both: the longevity of an open protocol, with the convenience and polish of a proper forge.
> It’s the first truly federated code forge — you can host code on your own servers and still collaborate seamlessly with others. Our approach is pragmatic: self-host just the parts you want, like Git hosting or CI runners, or all of it, including the API server and UI. Whatever you choose, you can still talk to everyone else.
The comment here from @icy emphasizes similar ability to let whomever host whatever parts of this they please:
> Hi! Tangled co-founder here. We're super stoked with this launch -- we've stabilized our AT Protocol lexicons, completely overhauled our architecture and UI, and made self-hosting the entire stack (or parts of it!) https://news.ycombinator.com/item?id=50306205
I don't see any mention of "their struggles around scaling centralized infrastructure". Is that just typical run of the mill low grade slander attack we see against atproto, more reflexive negativity? Or is there some actual interesting point here, that is perhaps even relevant to this tangled release?
They had the mostly-centralized offering up immediately, and now they're spending a lot of time making the parts easier to self-host one by one. What does that say about the protocol? And I assume Tangled also suffers from the general ATProto AppView problem, where your AppView has to consume all the content in the entire network even though 99% of it is likely uninteresting to you.
Code hosting platforms are probably much more disjointed than microblogging platforms, so something like ActivityPub, where servers only keep track of content that users are actually interested in, would make much more sense. It's a shame that the people working on ForgeFed and/or Forgejo federation seemingly aren't making much progress.
I think you fundamentally misunderstand how the protocol works.
First, Tangled has been entirely self-hostable since the beginning! We've just made it much easier to run for everyone.
Second, the "appview" is simply an indexer. It consumes records from the firehose and only those that it cares about. In our case, those are org.tangled.*. There's no reason we'd consume all of the firehose.
ActivityPub categorically does not make sense here because of how coupled identities are to instances.
> Tangled has been entirely self-hostable since the beginning!
As has BlueSky, as long as you had enough hardware to keep up with the firehose. The protocol heavily lends itself to centralization due to this design. I think this should be obvious by the fact that you haven't considered self-hosting to be the default case.
> In our case, those are org.tangled.*.
I suppose my wording was inaccurate, but this is what I meant by "the entire network". My indexer would have to consume "all of Tangled" instead of the things related to the repos, orgs and people I am interested in. Correct?
> ActivityPub categorically does not make sense here because of how coupled identities are to instances.
I feel like this is more of an inconvenience rather than a dealbreaker. Having to point people to a new account ID on the off-chance you lose your domain seems like much less of a problem than having to deal with ATProto's firehose architecture.
> As has BlueSky, as long as you had enough hardware to keep up with the firehose
We've already explained to you twice that this is incorrect, you don't need to do this. And that if you do want to do this for some ridiculous reason, just because, a $5 VPS can do it.
But you do acknowledge latter you are only talking about tangled records.
Currently very few people are worried about this problem, because all of the tangled traffic just isn't that big, and you can drop most of it very quickly with little impact. Relays do allow various kinds of filters though. You could use a service like Microcosm Constellation https://constellation.microcosm.blue/ that lets you find only records for your repos etc. That would work fine for finding the records pertinent to you!
No one does this right now, because the premise is faulty. There just really isn't any difficulty or cost of any note to having all the tangled records piped at you. Hopefully some day this gets worse, and that this becomes a good problem to have! One we already have tools (like constellation) to tackle.
In short: constellation is a (also easy to self host cheap light weight) service for reverse indexing, for finding records pertinent to you. Lots and lots of atproto apps use this for this kind of "find people making records about my stuff".
> for some ridiculous reason
A ridiculous reason, such as being able to interact with the federated world from my own server directly, and not through a third-party service. Why is the simple concept of going to your own server to interact with the entire network, getting content directly from your friends' servers, and receiving interactions (e.g. PRs) directly from someone else's server such an afterthought in ATProto?
The relay exists because it means running your own PDS server is very easy and cheap to do.
You can host your own data very cheaply. If you don't have to serve every single other server, and keep them all up to date, and can just pump some records out at aggregating relays, it makes it much easier for people to be online. You can run a massive PDS on a tiny ancient rpi.
Which is what activitypub doesn't do. It's protocols are very heavy and hard and stateful. Running mastadon or other servers requires peering with every other related server. And it's on you to deliver messages, to push, to track state and progress. All the network work is on you to get your message out. So your server with your data has a ton of work to do. This doesn't scale well. See again Paul's scaling with open social.
If you do want to run your own atproto services, yes, you need to pull from the network to see what is relevant, you need a relay server, you need a pull. And you can! Running those relays is very cheap! The protocols scale really well & is super well optimized to be efficient to do! You can just do it!
The one particular thing you keep trying to demand is what makes AP so hard, so difficult, and scale so very badly. It's the worst part of activitypub, it makes it super hard and super expensive. Switching from AP push model to the relay doing pull model is the secret sauce that makes everything super lightweight and scalable, and that makes sync so much easier.
You already could host either the web app, the knot (the repo), or the spindle (the runner) yourself, they were just harder to run yourself. I think it says a lot of good things that you can one by one pick which pieces of the system you want to host where!
It is a social network so I'm not sure what the issue about yes needing to read the network is. There are really good relays designed to help aggregate and filter the network. They can run on $5 VPS, processing the entire network, which is awesomely cheap, not just Tangled, which is awesome, what I would call a huge win. Paul has a talk on designing for scalability, about how it's possible to have such small boxes doing the network aggregation with so little resources, that appviews can then consume their interests from. solving for scale in open social networks, https://youtu.be/BoJnj2yPf14
The servers are definitely not keeping the whole network. The post talks to this some, but the new Knot2 retains a local copy of repo-relevant material (comments prs etc) in its own git tree, that collects what the network has done together, and makes it easy to ship/replicate, while also using that as a sync source to replicate back into atproto too, to make network visible.
Agreed that I really wish ForgeFed were showing much more clear progress. It seems like such a good idea, but it's been a number of years now and it's super murky what the progress is and where we are. Having >1 viable player in the decentralization of forges would be great. I haven't checked in a bit but I do read Forgejo release notes and my impression is it's been a very slow trickle.
> Having >1 viable player in the decentralization of forges would be great.
There are other projects other than Forgejo implementing ForgeFed. Forgejo itself has some federation support but AFAIK not based on ForgeFed. But they have a community contrib project that does.
- https://codeberg.org/forgejo-contrib/federation
- https://codeberg.org/activitypub-rs/activityforge
- https://frogue.vit.am/~ololduck/Frogue
Gitlab has a community epic to add ActivityPub support, but that needs contributors:
- https://gitlab.com/groups/gitlab-org/-/work_items/11247
When you say "they can run on a $5 VPS", do you mean that the entire server will be dedicated exclusively to processing ATProto data? I run an ActivityPub server on my $5 VPS that allows me to interact with everyone else on ActivityPub, and that VPS is hosting a lot more services than just that.
Yes I mean a $5 VPS that is processing all network traffic for the entire atmosphere.
Generally you don't have to do this! There's lots of people running relays that are open access. But if you want your own it's very cheap. List of relays: https://tangled.org/firehose.club/community-firehose-list
Can you also fit a PDS to host yourself on on that VPS? Yeah I expect that it would fit very easily. PDS's have a lot less traffic hitting them than ActivityPub, they can be way smaller. They are quite lightly loaded! Because... there are relays in front of them!