Making requests is inherently cheaper than delivering responses, even with caches. Efficiency improvements can buy a little time on a given resources but won't solve the problem of bot saturation now that everybody can spawn a custom bot in about 12 seconds and is being encouraged to do so.
Rate limits, blocking, and pay-per-use are the only roads out and even those might not last as models get better at hacking and masquerading.
The internet we want to use LLM's with is simply not one that can support LLM's, and with LLM's not going anywhere, the whole experience of the internet is going to be forced into some radically less open and more expensive paradigm.
Policies like this just represent the beginning of the transition.
We need a new non-commercial version of the internet. Free of bots, free of ads, ... you pay to access social media optimized to be interesting enough to be worth paying instead of addictive enough to keep you scrolling to show you more ads.
It may look impossible right now. But what is impossible for real is to continue as we are. The damage that internet does to society is increasing by the day while its value is reduced (economic value, social value).
> We need a new non-commercial version of the internet.
> you pay to access social media optimized to be interesting...
So is it commercial or non commercial?
Nothing is stopping you from creating a social network that is pay gated. Go build it. If you can't get anyone to sign up perhaps you'll realize it's not so easy as scapegoating addictive social media.
There is a whole cottage industry of people who legitimately make their living criticizing Facebook. It's a consumer software product. Yet few of these people seem to have their conviction extend to building an alternative that ever catches an audience. Why is that? Because addiction? Any other excuses?
It only proves your point because it wasn't successful in the end, but I really enjoyed the experience on app.net (a paid version of Twitter) while it still existed. Personally I think they lost it where a lot of these alternative platforms lose it - they try to build an ecosystem that is always coming soon and ends up scattered instead of focusing on one product
You'd need a force strong enough to prevent it from falling prey to this tragedy of the commons, and that force would need to be stronger than the incentives to commercialize it. And that's where plenty of contemporary scraping-based salaries lay.
Careful what you wish for. The ticket to entry will (have to) be a proof of identity, the ultimate nail in the coffin of privacy and anonymity on the web.Outside of that, chaos.
That's this internet. Nothing precludes you from dropping big tech and solely engaging with the "indie web"[0][1] if you so choose. You're free to build your own website with its own RSS feed, join webrings[2] of like-minded people, and engage organically.
[0] One of many similar initiatives, I'm sure. Not an endorsement.
Thinking about it, besides library access, the internet is already behind ISPs and requires funds to access. Having I2SP's does not solve the underline problem.
The TOR network is probably the closest thing to what OP is describing but that is filled with illicit material and has no safeguards. We all pretend we don't like internet laws but those rules keep criminals below the surface and normal users from accidently stumbling on it.
Anonymous KYC is the way forward. Legislation on preventing bots presenting as legitimate users should be implemented. Similarly to how the world has mostly made robo calling illegal.
I'm not so certain. I used to buy a broadsheet newspaper, which was full of ads. Now I pay a few hundred a year, and get the same newspaper online, with no ads at all.
So, the precursor to online media has already gone through this paradigm shift.
I mean, what you're saying is "If I pay for a product ads go away"
Which is partially true, but it only shifts the distribution of the problem. Once your service gains enough popularity network effects cause it to gain value. You have to worry about high priced buyouts of the entire service (great for the site owner, terrible for the users).
It's not, it just requires creativity. One example I can think of is limiting connectivity by distance between nodes. Something like Meshtastic seems unlikely to ever be commercialized in the same way that the Internet has been. Sure, you would lose some useful applications of the Internet, but you would gain other things.
Presumably but I wish they'd also focus on optimizing code/making pages fully cacheable instead of rate-limits and blocking
Making requests is inherently cheaper than delivering responses, even with caches. Efficiency improvements can buy a little time on a given resources but won't solve the problem of bot saturation now that everybody can spawn a custom bot in about 12 seconds and is being encouraged to do so.
Rate limits, blocking, and pay-per-use are the only roads out and even those might not last as models get better at hacking and masquerading.
The internet we want to use LLM's with is simply not one that can support LLM's, and with LLM's not going anywhere, the whole experience of the internet is going to be forced into some radically less open and more expensive paradigm.
Policies like this just represent the beginning of the transition.
The limits are for API requests, no? Or is this just an unrelated performance complaint?
Last I checked, they just blanket added `cache-control: max-age=0` to everything.
Loading https://gitlab.com/gitlab-org/gitlab is showing 14 API calls so presumably significantly more DB queries for a public page
We need a new non-commercial version of the internet. Free of bots, free of ads, ... you pay to access social media optimized to be interesting enough to be worth paying instead of addictive enough to keep you scrolling to show you more ads.
It may look impossible right now. But what is impossible for real is to continue as we are. The damage that internet does to society is increasing by the day while its value is reduced (economic value, social value).
> We need a new non-commercial version of the internet.
> you pay to access social media optimized to be interesting...
So is it commercial or non commercial?
Nothing is stopping you from creating a social network that is pay gated. Go build it. If you can't get anyone to sign up perhaps you'll realize it's not so easy as scapegoating addictive social media.
There is a whole cottage industry of people who legitimately make their living criticizing Facebook. It's a consumer software product. Yet few of these people seem to have their conviction extend to building an alternative that ever catches an audience. Why is that? Because addiction? Any other excuses?
It only proves your point because it wasn't successful in the end, but I really enjoyed the experience on app.net (a paid version of Twitter) while it still existed. Personally I think they lost it where a lot of these alternative platforms lose it - they try to build an ecosystem that is always coming soon and ends up scattered instead of focusing on one product
> f you can't get anyone to sign up perhaps you'll realize it's not so easy as scapegoating addictive social media.
Regulations, we need regulations for it to work. Capitalism is not going to solve a problem that capitalism has created.
> It's a consumer software product.
Please read: How Facebook contributed to genocide in Myanmar and why it will not be held accountable. - https://systemicjustice.org/article/facebook-and-genocide-ho...
> Why is that?
I value human live, I have morality and I try to be a good person and help society.
Who pays you to defend Facebook? Do you have shares? Do you have interests?
> Who pays you to defend Facebook? Do you have shares? Do you have interests?
Is this the best you can do?
> We need a new non-commercial version of the internet.
Except it's non-commercial, therefore valuable, therefore commercialized, therefore commercial.
You'd need a force strong enough to prevent it from falling prey to this tragedy of the commons, and that force would need to be stronger than the incentives to commercialize it. And that's where plenty of contemporary scraping-based salaries lay.
Careful what you wish for. The ticket to entry will (have to) be a proof of identity, the ultimate nail in the coffin of privacy and anonymity on the web.Outside of that, chaos.
[dead]
That's this internet. Nothing precludes you from dropping big tech and solely engaging with the "indie web"[0][1] if you so choose. You're free to build your own website with its own RSS feed, join webrings[2] of like-minded people, and engage organically.
[0] One of many similar initiatives, I'm sure. Not an endorsement.
[1] https://indieweb.org/
[2] https://en.wikipedia.org/wiki/Webring
This is fine when you hide in the dark forest, small and unassuming, but the moment something discovers you, then you get ate by a predator.
Your human engagement will attract said predators because it's a unique information signal.
Yeah but you lose anonymity unless you can devise a scheme to identify advertisers and bots separately from and without identifying real humans.
AT Protocol and PDS’. BitTorrent to share and distribute bundles of data.
Impossible to create
Also for better or for worse the ad-centric model to some extent has allowed more people to access information.
Gating everything behind paid (but with no ads) likely would hurt a significant amount of lower income users.
Thinking about it, besides library access, the internet is already behind ISPs and requires funds to access. Having I2SP's does not solve the underline problem.
The TOR network is probably the closest thing to what OP is describing but that is filled with illicit material and has no safeguards. We all pretend we don't like internet laws but those rules keep criminals below the surface and normal users from accidently stumbling on it.
Anonymous KYC is the way forward. Legislation on preventing bots presenting as legitimate users should be implemented. Similarly to how the world has mostly made robo calling illegal.
I'm not so certain. I used to buy a broadsheet newspaper, which was full of ads. Now I pay a few hundred a year, and get the same newspaper online, with no ads at all.
So, the precursor to online media has already gone through this paradigm shift.
I mean, what you're saying is "If I pay for a product ads go away"
Which is partially true, but it only shifts the distribution of the problem. Once your service gains enough popularity network effects cause it to gain value. You have to worry about high priced buyouts of the entire service (great for the site owner, terrible for the users).
It's not, it just requires creativity. One example I can think of is limiting connectivity by distance between nodes. Something like Meshtastic seems unlikely to ever be commercialized in the same way that the Internet has been. Sure, you would lose some useful applications of the Internet, but you would gain other things.