Waterfall can work great when: 1/ the focus is long-term both in terms of knowing that she company can take a few years to get the thing live but also that it will be around for many more years, 2/ the people writing the spec and the code are largely the same people.
Agile was really pushing to make sure companies could get software live before they died (number 1) and to remedy the anti-pattern that appeared with number 2 where non-technical business people would write the (half-assed) spec and then technical people would be expected do the monkey work of implementing it.
No.
Agile core is the feedback loop. I can't believe people still don't get it. Feedback from reality is always faster than guessing on the air.
Waterfall is never great. The only time when you need something else than Agile is when lives are at stake, you need there formal specifications and rigorous testing.
SDD allows better output than traditional programming. It is similar to waterfall in the sense that the model helps you to write design docs in hours instead of days and take more into account as a result. But the feedback loop is there and it is still the key part in the process.
I've live through both eras...
Agile, hardly any planning, write 3 times.
Waterfall, weeks of planning, write 3 times anyway.
The point is, people don't know what they want or are asking for, until it's in front of them. No system is perfect, but waterfall leads to bigger disasters.
Any real software (that delivers value over time) is constantly rewritten and that's a good thing. The question is whether the same people are rewriting it that wrote it and what percentage of that rewriting is based off of a spec or based off of feedback from elsewhere in the system.
> Feedback from reality is always faster than guessing on the air
Only if you have no idea what the results will be.
Professional engineering takes parts with specific tolerances, tested for a specific application, using a tried-and-true design, combines them into the solution that other people have already made, and watches it work, exactly as predicted. That's how we can build a skyscraper "the first time" and have it not fall down. We don't need to build 20 tiny versions of a building until we get a working skyscraper.
But when you build a skyscraper you don’t one shot a completed building that stays static its entire life - you build a set of empty floors that someone else designs & fits out, sometimes years after the building as a whole is commissioned, usually several times in the lifespan of the superstructure.
And in the fitting out there often are things that exist only to get customer feedback (of sales), such as model apartments, sample cubicle layouts etc.
So yes, you are right that engineering can guide us to building something right first time - the hard part from software perspective is usually building the right thing, no the thing right.
An interesting analogy I came across once but could never find again is that with software systems, we’re not building a building, we’re designing a factory that produces an output - the example was a mattress factory that took in raw rubber feedstock & cloth and produced mattresses.
Are you running a mattress factory? Or are you trying to run a hotel, and need mattresses, so you build a mattress factory? The "software industry" is that - dysfunctional with perverse incentives.
We should not be building the same software over and over and over and over. I've built the same goddamn app 10 times in my career. And I watch other people build it, making the same old mistakes over and over, like a thousand other people haven't already gone through this and could easily tell you how not to do it. In other engineering professions, they write that stuff down, and say "follow this plan" because it avoids all the big problems. Thank god we have a building code and not "agile buildings".
Agile sucks because it incentivizes those obvious mistakes and reinventing of wheels. Planning allows someone to stop and look up the correct way of building the skyscraper before it's 100 feet in the air with a cracked foundation.
> The only time when you need something else than Agile is when lives are at stake, you need there formal specifications and rigorous testing.
Lives are always at stake, given that we use software everywhere, and often in unintended ways, even outside its spec (isn't that a definition of a "hack"?).
People think of medical appliance software, space/air traffic software, defense systems or real-time embedded systems as the only environments where "lives are stake", but actually, in subtle ways, a violation of user expectancy (in some software companies, UX issues count as serious bugs) in a Word processor, Web browser or the sort command can kill a human.
Two real-life examples:
(1) a few years ago, a Chinese factory worker was killed by a robot. It was not in the spec that a human could ever walk in the robot's path (the first attested example of "AI" killing a human that I found at the time). This was way before deep larning entered the stage, and the factory was a closed and fully automated environment.
(2) Also a few years back, the Dutch software for social benefits management screwed up, and thousands of families just did not get pay out any money at all for an extended period. Allegedly, this led to starvations (I don't have details - but if any Dutch read this, please share), and eventually a whole Dutch government was forced to resign over the scandal.
That's a very narrow definition of engineering. What about property? Sensitive information?
It's a fine "whoopsie-doodle," when your software erases the life savings of a few thousand people. "We'll fix that in the next release," is already too little, too late.
"Waterfall is never great."
The only software I ever worked on that delivered on time, under budget, and with users reporting zero bugs over multiple deliveries, was done with heavy waterfall. The key was knowing in advance what we were meant to be making, before we made it. This did demand high-quality customers; most customers are just not good enough.
This is correct. Agile is control theory applied to software engineering.
The plant to control here isn't something simple like a valve. You're performing cascaded control of another process where the code base is the interface to the plant you're controlling.
Royce's Waterfall paper that coined the term is all about a series of short feedback loops, like water bouncing off of rocks on the way down.
I spent my career building software for executives that wanted to know exactly what they were going to get and when because they have budgets and deadlines i.e. the real world.
Mostly I’ve seen agile as, let’s do the same thing 3x we could have done once if we spent time on specs. The key phrase here is “requirements analysis” and if you’re not good at it either your software sucks or you’re going to iterate needlessly and waste massive time including on bad architecture. You don’t iterate the foundation of a house.
I see scenarios where Agile makes sense (scoped, in house software, skunk works) but just like cloud, jwts, and several other things making it default is often a huge waste of $ for problems you/most don’t have.
Talk to the stakeholders. Write the specs. Analyze. Then build. “Waterfall” became like a dirty word. Just because megacorps flubbed it doesn’t mean you switch to flying blind.
> The key phrase here is “requirements analysis” and if you’re not good at it either your software sucks or you’re going to iterate needlessly and waste massive time including on bad architecture. You don’t iterate the foundation of a house.
This depends heavily on the kind of problem you are trying to solve. In a lot of cases requirements are not fixed but evolve over time, either reacting to changes in the real word environment or by just realizing things which are nice in theory are not working out in practice.
You don’t iterate the foundation of a house because we have done it enough times and also the environment the house exists in (geography, climate, ...) is usually not expected to change much. If that were the case we would certainly build houses differently than we usually do.
The key first step here is "Problem Identification" - the number of times I have seen where it is part way through a development, and only then it starts to become obvious that even if the specifications were good, they were not solving the right problem.
Users have a habit of describing their problem as the solution they think they would like to have, often being disastrously far from the actual need.
> making it default is often a huge waste of $ for problems you/most don’t have.
It's the opposite — knowing the exact spec of your program up front is vanishingly rare, probably <1% of all projects. Usually you have no clue what you're doing, just a vague goal. The only way to find out what to build is to build something, toss it over to the users and see what happens.
No developer or, dear god, "stakeholder" can possibly know what the users need. Asking the users up front is better, but still doesn't help much — they don't know what they want either.
No plan survives first contact with the enemy and there's no substitute for testing — reality is far too complex for you to be able to model it up front.
> You don’t iterate the foundation of a house.
You do, actually. Or rather, we have — over thousands of years we've iterated and written up what we've learned so that nobody has to iterate from scratch for every new house anymore. It's just that our physics, environment, and requirements for "a house" doesn't change constantly, like it does for software and we've had thousands of years to perfect the craft, not some 50 years.
Also, civil engineers mess up in exactly the same ways. Who needs testing? [1]. Who needs to iterate as they're building? [2].
[1]: https://youtu.be/jxNM4DGBRMU?t=397
[2]: https://youtu.be/jxNM4DGBRMU?t=837
> knowing the exact spec of your program up front is vanishingly rare, probably <1% of all projects
I don't have anything useful to add, but both of you speak and write with conviction from your own experience and perspective yet to refuse that the situation might be different from others.
"Software engineering" is a really broad field, some people can spend their whole life working on projects where everything is known up front, others the straight opposite.
Kind of feel like you both need to be clearer up front about your context and where you're coming from, otherwise you're probably both right, but just in your own contexts.