This is the right way to deliver software.
Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs.
In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.
Its a yes if you do not know the domain space, query patterns well enough and also if the cost of optimization or time for optimization may have detrimental impact to business. In this case it most likely means that the crowd in the room did not anticipate much on this in early phases and no one in the room pointed these things out. The irony is that these performance and disk numbers are heavily discussed as a part of system design interviews.
> In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivia
This is a misconception when you including roll out as a part of the change too, changing data once its running in production is hard, changing the data structure is even harder and when you talk about making changes in cache which is at the hot path its probably the hardest. Looking at the graph at the end it looks like it took them 4+ months to roll out the changes after optimization.
“changing data once its running in production is hard, changing the data structure is even harder”
100% agreement on this. There are a class of optimizations that can happen transparently. Those can happen at any time, and are fine to defer. Not all profiling and scalability improvements fall into this bucket. Some are very expensive to roll out, and ignoring these concerns can cause huge headaches down the line. Not fun to hear, but it’s definitely true. Even with LLMs, this can still be a huge challenge.
> This is a misconception
Pretty sure they were joking.
I do not think Cloudflare was a less-than-peers optimized product when they launched. This is one of their blog posts which describes taking one aspect even further.
I think Cloudflare became big only because they were so much more optimized than others that they offered some services for free that others were not offering. If running costs are high, you only burn (VC) cash and then you exit.
First version was a three layer VM monster, with the main entrance done as HTTP proxy written in PHP.
Or optimize a bit earlier and prevent having to scale out to a bazillion systems.
The way I usually prevent having to scale out to a bazillion systems is never getting more than 10 users.
And that's why I charge $10,000,000/user/mo.
I wonder why Cloudflare didn’t think of this
This is Broadcom's business model
Quite. I was a VMware fanboi (25+ years, man and boy)
I still look after a few VMware estates and a lot of Proxmox ones (that used to run VMware).
Hilariously, VMware is described as "enterprise class", which I can only conclude means MVP and a bit wanky.
Today I repaired a Proxmox HA + Ceph node using boring old normal Linux skills and as it turns out I have 30 years of those. Part way through a remote v8 to 9 upgrade I think I lost comms due to using OpenvSwitch for networking and despite using tmux for the upgrade session. Anyway, the Proxmox ISO was useless for rescue but the classic systemrescuecd worked nicely and I could run dpkg in a chroot.
VMware "used" Linux and never really gave back. I don't miss fixing vCentres and all the other nonsense that "Enterprise" wankery has foisted on me over the years.
When I was first getting into software dev, I thought 'enterprise' meant 'industrial grade', 'powerful', 'high-performing'.
Then I actually met some enterprise software, and realised that it means 'expensive', 'bespoke', 'one-off', and usually 'janky'.
Enterprise means it has SSO and a support contract
It means you are paying for a support contract. Whether you actually have one time will tell.
Lol, so true
Enterprise quality software is a just a DOS application, probably written for dBase III, that has been rolled forward to the present day.
Those old school systems are often much more stable than any newer systems. Autozone looks to use something like that and I’ve never seen them have issues as a customer.
To me it always meant needlessly complex and overspecced for what's needed. I think probably due to Java's enterprise years.
Why solve the problem directly when you can abstract everything away into FactoryFactoryImplementationInterfaceFactorys, and have something that is both a memory-hog and completely unassailable to any normal programmer seeking to understand it or make changes?
The art of premature optimizations
The art is in knowing how to write software that doesn't perform like shit without doing all the work of measuring and refining. If you can save $100k in hardware costs in a couple days by just knowing what you're doing, that optimization is not premature.
The Art of Production
pro move. made my evening.
You're never going to get promoted with that attitude!
I'm joking...but not entirely. It sounds impressive on a promo packet when you say you've saved 100 TB of RAM / $$$ through whatever technique. But it sounds a lot less impressive when you say if this system grows to this size in x years, I will have saved 100 TB, especially when no one yet knows how large the system will really be in that time or what the cost of RAM will be. I dunno, maybe if you say that x years ago, I made a decision that now is saving us 100 TB, that's kinda impressive, but you're also getting credit for it x years after you did the work. It also doesn't have the implication that it must be inherently complex/hard because some other smart person chose the other way. And there is a bias to care more about recent accomplishments. So I don't really think it'd be valued the same at all.
Also, in general big tech (at least Google) prefers growing the userbase over improving efficiency. Periodically efficiency is rewarded, e.g. when RAM cost suddenly balloons or some big must-have feature has suddenly used up capacity planned for something else. You get rewarded for doing efficiency work on demand, not eagerly.
I once got a $100 peer bonus for finding 100,000 cores that were essentially stranded by an accounting error in another team's migration script.
Remember that everything has an opportunity cost. Running a lot of servers might cost $10 million annually, but if the product team had to choose between a project that would recoup $5 million of that vs. an opportunity to earn $50 million ARR for the same amount of work, the logical answer would be obvious.
That's assuming the ops team has infinite capacity.
It was already reasonably lean. If they had 10 bazillion systems, they now need somewhere between 6 and 8 bazillion systems.
You can build foundations that aren't extermely optimal but have future optimisations in mind.
This assumes that you have plenty of cash to burn in the process, which is approximately correct for VC-backed ventures, and for offshoots of large corporations that play a lomg game.
Make it work, make it fast, refactor
> start generating profit, and then you can start optimizing your costs
Good thing they jumped on that as soon as they were profitable instead of burning cash. Oh wait...
I think a distinction to draw here is that Cloudflare had relatively large capital raises and were almost immediately profitable¹. They had the luxury of throwing away money. Judicious optimisation makes sense for scrappy start-ups, especially when trivial optimisations like these could easily be farmed off to an agent.
¹ https://timeline.www.cloudflare.com/
This reasoning assumes you have access to infinite runway. You don't.
Exactly, and you need to start turning a profit before the end of that runway. Even if that means running code that is suboptimal.
i suppose you could say the same about buying a house. just make that initial 300.000 and from there on out its easy. everything looks better on paper
My house is a ~700 sqft. condominium, gov. subsidized for lower income individuals, and even my mortgage is more than 300k… maybe I’m just basing my info off of coastal city prices, but is it possible to buy a reasonably nice home located in a reasonably nice amerikkkan city… for $300k in 2026?
That would buy you thousands of square feet and often several acres within 20 minute drive to a lot of US city downtown areas.
Can you ground the discussion by mentioning what you think these cities are? Taking Columbus, OH as the most average of American cities and a 20m isochrone map from city center, there are currently 0 parcels for sale with 3+ ("several") acres under $300k. There are a few within 30m drive, one of which even has a possibly habitable structure. The rest are bare agricultural land you'd need additional investment to actually live on.
Not from what I've seen. The desirable neighborhoods are 500-600k. Suburbs outside major city
Desirable neighborhoods are by definition expensive. The trick is to find a neighborhood you like where your home can just be a home and not a top-heavy investment.
Acknowledging this isn’t always easy or possible, but just pointing out that this is a self reinforcing problem.
> The trick is to find a neighborhood you like where your home can just be a home and not a top-heavy investment
I meant desirable for me to live there, not as an investment. Who wants to buy a home in place they don't want to live?
You didn't read what I said.
I mentioned acres of land. You normally don't have multiple acres of land in the suburbs.
Well then you didn't read parent's comment
> is it possible to buy a reasonably nice home located in a reasonably nice amerikkkan city… for $300k in 2026?
Who wants acreage? We want homes.
> is it possible to buy a reasonably nice home located in a reasonably nice amerikkkan city… for $300k in 2026?
What does "city" mean to you? For some, it's 500 people, or 5,000. For some, it's 5 million. Define that first. The US is a big place, and I know people that don't live within 50 miles of another human.
Otherwise:
https://www.zillow.com/homedetails/424-Olive-St-Kansas-City-...
4 bed, 3 bath, 1,580 sq ft, beautiful! $342,500, built in 1900.
https://www.zillow.com/homedetails/3508-N-College-Ave-Kansas...
4 bed, 4 bath, 2,295 sq ft, $365,000, built in 2022.
That’s 350, which you can’t buy for 300..
This reasoning is largely centered around the runway being finite. You obviously can't have costs so high you are making a huge loss, but also there's little value in improving margins past profitability until you actually have a stable segment of the market.
we are all perfectly smooth, round, and filled with an incompressible liquid
Every startup is one bet in a Martingale strategy played by the class of people who remain solvent when you bust.
The median return for a startup is $0. Take care when trying to extrapolate cause and effect.
Only if you have loads of capital
> Produce working product first, validate the idea, stabilize the business, start generating profit,
not everybody is so lucky to be able to go in that order? The first part requires upfront capital/investment?
So obviously you start at optimization
[dead]