These kinds of local-first syncing web apps are really interesting and can be really useful, but I think the premise is somewhat wrong.
"A few milliseconds is all it takes to update an issue in Linear. A traditional CRUD app doing the same thing takes about 300ms."
"Any data sent between the client and server costs hundreds of milliseconds."
There’s no solving the problem of a large RTT between an HTTP client and server when it’s due to the speed of light.
But what you can do is locate the backend near users and make sure it’s fast.
For example, it’s very possible to run a web app backend within ~10ms RTT of most users and have the backend render responses within ~10ms too.
In other words, you can absolutely create a traditional CRUD app where doing the same thing takes more like 30ms, not 300ms.
Thank you. Was beginning to feel like I was taking crazy pills seeing people claim that 300ms is fast when 30ms has been the target TTFB for as long as I can remember. Maybe Linear takes more time on the backend for totally valid reasons and it needs some help from the frontend, but that's not generalizable, and every bit of JS comes with its own costs.
When I started in this business, "sub-second" responses were the goal.
> it’s very possible to run a web app backend within ~10ms RTT of most users
Only if your users are all located quite close to each other, or (sadly very common) you only care about making it fast for US users and screw everyone else.
(Of course you can have "intermediary backends" around the world on a CDN's edge network or similar, but at that point you're paying the same complexity cost as this style of putting the "intermediary backend" on the client)
10 years ago, I was headlining a project where we had strict performance requirements (ssr, php mind you) - the target for any operation except login was 30ms, and any endpoint taking more than 60ms in one of the devs server would have to have explicit approval. Add the nework RTT if you dont want to have a local backend, and for most geographies is still well under 300ms. Fun fact, we actually designed a system that could be easily replicated between regions and/or perform internal routing, leveraging the operator network. 3 AWS regions would make the RTT of a request well within <100ms on average for >80% of the world population. Requests were mostly "instant" - the big trick that did it (at the time) as to avoid reflow in the browser. Funny how the spotlight is now on doing stuff that "solves" that problem with the technology that was designed to solve that problem.
> Only if your users are all located quite close to each other, or (sadly very common) you only care about making it fast for US users and screw everyone else.
Very true, mostly because it's pricey to do so--or more specifically, putting a server in DCA is $.005/user, putting a server in JNB is like $.50/user sooooo....
Sometimes this is a "so what", other times it has huge implications for app architecture.
> it’s very possible to run a web app backend within ~10ms RTT of most users and have the backend render responses within ~10ms too.
What are you talking about? The only AWS region < 10ms away from us-east-1 is, err, us-east-2:
https://www.cloudping.co/
us-west-1 is 60ms away. eu-centra-1 is 100ms away. asia is 200ms away.
and this is datacenter-to-datacenter traffic. Actual latency over the public internet to residential providers is far worse.
Your database needs to be in exactly one region. So no matter where you put it, the majority of uses on earth are going to be > 100ms away from it.
It doesn't matter where the endpoints are, because the endpoints need to talk to the database to read and write data. Thinking you can replicate some data closer to the users? Congrats you are now the proud owner of a "local-first syncing" database. This replicated database you use (either of your own design or off the shelf) will have all the same problems of client-side syncing. Except you'll still have significant network latency.
There is no getting around physics. You either have quarter-second commits for most users or eventual consistency (aka syncing). Those are the options.
"Your database needs to be in exactly one region. So no matter where you put it, the majority of uses on earth are going to be > 100ms away from it."
You're assuming a single global database, which ignores the many alternatives.
I’m not. See: https://news.ycombinator.com/item?id=48440209
These systems are designed for the very common case of a global user base. If you have geographically centralized users, you can maybe do something simpler. That is rare in my experience - basically all of our customers have users worldwide. They typically don’t even know where their users are so making ux tradeoffs based on that feels really risky.
But maybe my experience is different than yours. One of the amazing things about the software ecosystem is how big it is. Everyone thinks their view is the common case.
you can home tenants in a data center close to them, run a copy of your app in each region including the datastore. keep a central db for accounts, billing, etc but user content is easy enough to shard regionally.
taken to extreme, cloudflare durable objects & workers let you place data very close to a tenant automatically; but you lose total write throughput on top of sqlite.
This breaks down when someone goes on holiday to Greece for a week, and the RTT over the airbnb wifi is 5 seconds.
Optimistic updates on the frontend are probably simpler too.
oh for sure you start with client side cache & optimistic updates. but u need low latency / regional backend for multiplayer to feel good. I didn’t realize who i was replying to, aaron is probably one of the few people who think about sync engines more than me. anyways we do both at notion and of course we did local cache first client way before we did multi region at Notion.
But this is kind of meaningless unless the tenants themselves are in one geo. Take linear as an example, this strategy works as long as your company that uses linear is all colocated in one area. As soon as you have remote people it falls apart.
But it does mean you gracefully degrade so the majority of the company sees the target latency <100ms and the rest of the company sees "not geo-optimized" latency.
Only in the case where there is such a majority of the company that is tightly geolocated.
Again, AWS latency us-west-1 to us-east-1 is 70ms. That's absolute best case for one round-trip that does absolutely no work. And it's ignoring the case of anyone outside of continental US.
Add in actual server-side work, db interactions, and contention - and you're quickly looking at hundreds of ms.
You don't even need your backend that close if your server is fast enough. Streaming HTML immediate mode is pretty good. See this demo (server is in Germany and runs on a potato uses no optimistic updates, eveb scroll round trips) [1]
Honestly client side animations go a long way to masking latency too.
- [1] https://checkboxes.andersmurphy.com/
You can locate an "intermediary backend" on the client, write outstanding mutations into local storage, have a background worker send it to the backend, with necessary retries, etc. At worst, the background worker would put out a message about a failed update which the UI tread would receive and show.
But the happy path stays lightning-fast.
Sure but there's a ton of complexity in any kind of local-first syncing solution. Often the solution is CRDTs.
My point above is that the simple solution ("traditional CRUD app") is actually viable even when the goal is very low latency.
100% agree. A traditional CRUD app (like Linear) can be made pretty low latency without local-first. The complexity is not worth it.
How would you do it? The speed of light being what it is, and tail latencies being large unless you over provision wildly, you keep facing hundreds of milliseconds of reconciliation time, if you want guaranteed synchronous result.
I said it in another thread. Yes, of course we cannot match the local store interactivity. What I mean is coast to coast RTT is 65-80ms. And the server can be optimized to return back basic operations (adding a comment, a new ticket, reading one back etc) can be done within tens of milliseconds, keeping the entire thing under 100ms coast-to-coast. If one can colocate servers with users, it becomes even less. I'd rather to server side "reconciliation" than do both (because serverside thing doesnt go away even with localstore, it is just deferred).
Can you do this where you need to have a database shared between all these edge backends?
If your service shares state globally across all users (like a social network) not really. If individual customers are mostly centered around one geographic location and their data isn't shared with other customers, yes.
Even social networks tend to shard their DBs geographically. It's obviously not 100% effective (I can always add a friend on the other side of the world), but it significantly improves the average case (the majority of my friends will have gone to school/university/worked in the same geographical locations I did)