> How you manage to drag query planners into routers into compilers, how are these chasing one dream or fungible or even barely correlated, I don't even know.
Let me expand.
In component-based UIs you have components that fetch their own data. Declare their data dependencies if you will. This makes them independent and easy to move around.
You can blindly take a component that renders, say, the user’s name and avatar and move it anywhere in your app. It’s going to know how to recognize the current user, fetch their data, and render what it needs. This is great for reusability and flexibility.
Now what happens if you have a bunch of these components on a page? You get lots of uncoordinated data fetches. Each component is independent so it doesn’t know about the others so they can’t coordinate.
But who does know about them? That’s right, the router knows. So the router can make a more coordinated data fetch and get data for all those components.
This is the same job as a controller in the classical MVC approach where you write data fetching (sql queries) in the controller so data is ready when you access it in the template (jinja or whatever). Exact same problem, different naming conventions.
And just like with ORMs you have the same N+1 issue where your template pokes an attribute that was not prefetched and triggers a bunch of data fetches (sql queries). So you end up doing lots of tedious work to always hoist up the joins into your controller to trick the ORM into having that data preloaded.
Wouldn’t it be great if somehow your ORM could understand all the render logic and write its own single query that gets all the data? Yes it would. Just as wouldn’t it be great if your router could understand all the fetches your UI components will do and make one big request in the beginning? Ofc.
Now here’s how this relates to query planners: Wouldn’t it be great if you could just declare what data you want and the computer figured out how to read just the right amount from disk, join together the relevant bits, and gave you the output? Without you having to write a bunch of manual disk reading logic, loops to clean and correlate the data, and while also minimizing the amount of disk access manually? Of course it would.
Why should it be the router? A saner option is to let the parent component coordinate the data fetches for the child components. What you are suggesting is way out of the normal responsibility of a router.
The router is just the most parenty component. It’s the component that takes some input (like a url) and uses what is essentially a giant switch statement to decide what gets rendered.
You could justify putting anything and everything in the router with this argument.
> Now what happens if you have a bunch of these components on a page? You get lots of uncoordinated data fetches. Each component is independent so it doesn’t know about the others so they can’t coordinate.
No. You write a data layer that deduplicates requests. And since you typically need state management for fetches for UX, why not roll it all up into a tidy hook?
https://tanstack.com/query/latest
I guess the lynchpin is under-engineering at the request layer "forcing" massive over-engineering at the routing layer.
-
By the way, your self-answered rhetoricals are really really off the reservation in terms of relevance so I'm not even going to go there other than pointing out that a bad analogy is worse than no analogy.
You're trying to explain request waterfalls. Just say request waterfall. Look up the term if you have to, it's rare that you need to invent a definition for a basic and common problem.
> You're trying to explain request waterfalls
So conceptually what's the difference between a request waterfall and an N+1 query issue?
So practically what's the benefit of mentioning "Query planners inside a database" instead of request waterfall?
To say, by analogy, that coordinated data fetches are hard and we’ve been working on it for a long time in different contexts.
Analogies help me understand stuff and borrow lessons others have learned.
This just means that maybe the entire independent component concept is bad design. You tried to solve a problem by making it more complex so you could build apps like legos. What we had was a data layer or a parent component that handled this coordination. Apps were still built and still worked well. Browsers were not begging for forgiveness due to performance issues.
Also, before this, I've always been able to easily write the components you mention, easily I should say, without any of this N+1 madness. So why did components need to become their own apps inside a larger app?
Soon you're going to tell us that those components need their own infra, all pulled from different CDNs, all using separate auth and security contexts, and all will need to be sandboxed by the browser so they cannot access each others cookies. And that this is all needed and important and an improvement.
> Now what happens if you have a bunch of these components on a page? You get lots of uncoordinated data fetches. Each component is independent so it doesn’t know about the others so they can’t coordinate.
This is solved by tools like tanstack-query. Ten components using the same request will only trigger one API call, and the response will be shared.