I've re-read this comment at least 5 times and feel like I'm having a stroke reading it each time. And something similar happens really often when I enter the hype-driven side of React these days..
I do wish I had a more useful critique, and I'm not even trying to be mean (or boorish as it were) but you're rolling so many things up into each other that there's no useful way to interpret the statement. At least, not without ending up giving you a great chance to just say "no no no you completely missed what I'm saying" and then coming up with a new equally dizzying statement.
How you manage to drag query planners into routers into compilers, how are these chasing one dream or fungible or even barely correlated, I don't even know.
-
It's awful and sad how tech is one of the few fields that just insists on making things worse for itself. You can walk into McDonalds and ask how the process can be improved, and I guarantee every suggestion is targeted at making their jobs easier in a way that at least superficially aligns with making the service quicker, which is something the company does care about.
In tech you ask and someone goes on a poetic journey about how the paper cups, the drive-thru speaker, and the coffee machine are all chasing the same dream, and also we need a $100,000 espresso machine that takes 10 minutes of intense labor to brew a shot because then I'll be qualified to operate a $100,000 espresso machine at my next job which pays better than McDonalds.
We did not figure out how to brew coffee before, that was all wrong and we needed to make the process at least 10x more complicated.
> How you manage to drag query planners into routers into compilers, how are these chasing one dream or fungible or even barely correlated, I don't even know.
Let me expand.
In component-based UIs you have components that fetch their own data. Declare their data dependencies if you will. This makes them independent and easy to move around.
You can blindly take a component that renders, say, the user’s name and avatar and move it anywhere in your app. It’s going to know how to recognize the current user, fetch their data, and render what it needs. This is great for reusability and flexibility.
Now what happens if you have a bunch of these components on a page? You get lots of uncoordinated data fetches. Each component is independent so it doesn’t know about the others so they can’t coordinate.
But who does know about them? That’s right, the router knows. So the router can make a more coordinated data fetch and get data for all those components.
This is the same job as a controller in the classical MVC approach where you write data fetching (sql queries) in the controller so data is ready when you access it in the template (jinja or whatever). Exact same problem, different naming conventions.
And just like with ORMs you have the same N+1 issue where your template pokes an attribute that was not prefetched and triggers a bunch of data fetches (sql queries). So you end up doing lots of tedious work to always hoist up the joins into your controller to trick the ORM into having that data preloaded.
Wouldn’t it be great if somehow your ORM could understand all the render logic and write its own single query that gets all the data? Yes it would. Just as wouldn’t it be great if your router could understand all the fetches your UI components will do and make one big request in the beginning? Ofc.
Now here’s how this relates to query planners: Wouldn’t it be great if you could just declare what data you want and the computer figured out how to read just the right amount from disk, join together the relevant bits, and gave you the output? Without you having to write a bunch of manual disk reading logic, loops to clean and correlate the data, and while also minimizing the amount of disk access manually? Of course it would.
Why should it be the router? A saner option is to let the parent component coordinate the data fetches for the child components. What you are suggesting is way out of the normal responsibility of a router.
The router is just the most parenty component. It’s the component that takes some input (like a url) and uses what is essentially a giant switch statement to decide what gets rendered.
You could justify putting anything and everything in the router with this argument.
> Now what happens if you have a bunch of these components on a page? You get lots of uncoordinated data fetches. Each component is independent so it doesn’t know about the others so they can’t coordinate.
No. You write a data layer that deduplicates requests. And since you typically need state management for fetches for UX, why not roll it all up into a tidy hook?
https://tanstack.com/query/latest
I guess the lynchpin is under-engineering at the request layer "forcing" massive over-engineering at the routing layer.
-
By the way, your self-answered rhetoricals are really really off the reservation in terms of relevance so I'm not even going to go there other than pointing out that a bad analogy is worse than no analogy.
You're trying to explain request waterfalls. Just say request waterfall. Look up the term if you have to, it's rare that you need to invent a definition for a basic and common problem.
> You're trying to explain request waterfalls
So conceptually what's the difference between a request waterfall and an N+1 query issue?
So practically what's the benefit of mentioning "Query planners inside a database" instead of request waterfall?
To say, by analogy, that coordinated data fetches are hard and we’ve been working on it for a long time in different contexts.
Analogies help me understand stuff and borrow lessons others have learned.
This just means that maybe the entire independent component concept is bad design. You tried to solve a problem by making it more complex so you could build apps like legos. What we had was a data layer or a parent component that handled this coordination. Apps were still built and still worked well. Browsers were not begging for forgiveness due to performance issues.
Also, before this, I've always been able to easily write the components you mention, easily I should say, without any of this N+1 madness. So why did components need to become their own apps inside a larger app?
Soon you're going to tell us that those components need their own infra, all pulled from different CDNs, all using separate auth and security contexts, and all will need to be sandboxed by the browser so they cannot access each others cookies. And that this is all needed and important and an improvement.
> Now what happens if you have a bunch of these components on a page? You get lots of uncoordinated data fetches. Each component is independent so it doesn’t know about the others so they can’t coordinate.
This is solved by tools like tanstack-query. Ten components using the same request will only trigger one API call, and the response will be shared.
The promise of newer tech like Next, InertiaJS, and Phoenix live view is that you don't need to build an API. The idea is: why are we spending all this time defining interfaces for coordinating data retrieval and unmarshalling, when the frontend already knows what the user needs to see?
They all take different approaches, but the core idea is the same: close the gap between the frontend and backend, simplify your state, and keep good performance.
You're grouping wildly unlike things.
Backend for Frontend is one thing, frameworks that hoist frontend state to the server (LiveView, Blazor, etc.) are another.
And none of the benefits you're describing are tied to RSC or anything that OP was banging their head into. React Router offers loaders that run on the frontend and have largely the same ergonomics.
tRPC/oRPC also remove most of the ceremony around building an API while also skipping the downside of implicitly creating a loosely typed API for your frontend.
But then again, this isn't really that fruitful of a conversation. I think even by the tone of this comment I can tell you're bought in, and I'm not. People who aren't bought in are just looking for reasons to complain, people who are are busy building. Right?
Yes all their implementations are wildly different, however their motivation is the same: maintaining complex front end state and backend state and then syncing it via a manually defined API is too complex. That, combined with the problem of how do we solve SSR in such a way that minimizes complexity while still maximizing front-end reactivity.
Live view says "render it all on the server and then just constantly sync with an open connection". Now we have that reactivity, but we still have SSR AND we don't have to meticulously craft an API. Next says "render it on the server, or maybe the client, we'll figure it out and automap the data". Inertia says "okay kind of like next, but you have to define some stuff a little bit more. But we'll render react or vue on the server or client, too".
And, for the record, I hate next, and I think they're approaching it from the wrong direction. I think inertia makes more sense, and ultimately live view is the "end state" of this concept.
the niche that Next dominates rightfully is these latency optimized but highly dynamic and parallelizable sites, minimal "per user" state lives close to the user, it's vertically integrated, and gives tools to frontend-oriented devs.
React expanded into server-side, it's messy now, but not that complicated.
Things look like the SpaceX rocket motor with all the diagnostic stuff still on, but in a few revisions it'll likely look (and feel) more sleek.
...
And it's true that the end-to-end experience matters at restaurants, be they fine dining 3 thousand Gault Milleu stars or a worldwide franchise with so perverse incentives built in that it should be 18+ anyway (and not just because of the hyper-palatable sauces).
...
And the argument is that these things matter especially when you have a lot of users.
...
That said the people responsible for docs at NextJS should be sent to the fullstack mines.
The niche Next dominated before they started chasing the RSC dragon was scaling SPAs with complex frontend state.
React Query and atom based state management offered nice alternatives to Redux/Context hell, and Next could have just hung up major architectural refactors for QoL updates.
They want to dominate the headless Shopify/CMS niche, but that doesn't have to mean dragging Next.js along for the ride. It was a business/branding decision, not one that actually benefits most users of what they had already had.