> for me it seems like you have to actually run into problems over and over and figure out how to avoid the problems
This shows how immature the field of software engineering is. Imagine bridges or houses were built like that. Or your surgeon was trained like that.
Over time, we hopefully develop estblished norms, but at the moment, things are too much in flux. Put 5 sw engineers in a room, pose a problem and you will get not just 5 different solution proposals, but there will likely be strong disagreements on which approach is a good one.
"I recognize a good solution when I see it" is just not good enough for a serious engineering discipline.
> Imagine bridges or houses were built like that. Or your surgeon was trained like that.
While I don't disagree with you in general, this does feel a bit off.
By that logic you can call the field of music immature, and all of the arts. I think the difference is that its easy to experiment without high costs.
I genuinely think that if building bridges was cheap and quick, the fastest way to learn was to try...
I generally agree with your point on ease of experimentation, but if we insist on calling it software engineering, then maybe the field needs to adhere to engineering principles, as the GP highlighted.
I believe part of engineering isn’t over-engineering for the task at hand as well. If the costs of a “failure” are low/zero then the right thing can be to move quickly expecting some problems.
I think the field could get better at knowing when costs are low (eg sometimes scalability, cheaper to change a database choice than rebuild a bridge) and where the costs are sometimes very high (eg security).
Notice OP says "software design". Design is still an art, even in mechanical engineering.
Engineering applies to looking at a design and proving something about it.
The term "design" is ambiguous here.
When design is a process to build something pleasing (like in music or painting), then yes, it's an art, and you can rightfully have different opinions about outcomes, sometimes diametrically opposed opinions, which is all not just fine and accepted but inherent in the field and a required ingredient.
When design is about building something functional, then it's part of engineering. You can objectively measure whether it does satisfy the requirements. That's where serious engineering disciplines are and where software engineering needs to aim for if it wants to keep engineering in its title without getting laughed at. As long as programmers consider themselves part artists, that's hard to achieve, but as the field matures, there is hope.
Note that some fields are trying to be both. For example architecture. There it's solved by making a more or less clear distinction between the pleasing art part (that's what the architect does) and the functional engineering part (structural engineer). Both roles are not being conflated. In software, we still do that.
> Imagine bridges or houses were built like that.
Bridge building is a lot more conservative when it comes to taking risk in the construction, but that is how we build bridges and lots of bridges collapse because of similar causes:
An average of 128 bridges collapse annually in the United States. More than 17,000 bridges in America are considered "fracture critical" (vulnerable to collapse from a single impact).Engineering doesn't mean all processes are flawless and surely not that all humans participating are perfect.
The collapsing bridges in the US are mainly caused by insufficient maintenance, which is not an engineering problem but a political one. And the vast majority of collapses happens with bridges that are known to be unstable and already blocked for traffic. The engineering part did that, as if it screams "told you so" at the politics which doesn't allocate sufficient funds for maintenance.
> Imagine bridges or houses were built like that
If they could afford experimenting and have a few bridges collapse before they get it right with no significant negative consequences IMHO it wouldn’t be the worst way to learn.
Maybe even more so for surgeons, being able to experiment and fail in a risk free environment seems like a good thing.
> This shows how immature the field of software engineering is. Imagine bridges or houses were built like that. Or your surgeon was trained like that.
It's not that software engineering is immature, it's just more dynamic.
We are not the surgeon, we write the surgeon. We write a surgeon to fix a broken leg. Once that is done, we don't have to fix another leg. Now we need to reattach a finger. Once that is solved, maybe replace a kidney.
You cannot repetitively train or have strict rules for that, because every time it's something new. You need to have broad knowledge and experience to be able to fight the next unknown challenge. It's unknown because it's never been done before, or it has been done but your competitor will not reveal the details.
Building bridges or being a surgeon sounds very boring to me, since it's always the same (maybe some minor variants). Building software? Very much not the same.
Most swe is just crud
> This shows how immature the field of software engineering is. Imagine bridges or houses were built like that.
You're forgetting a key difference between software and the physical world.
The blessing and curse of software is that there are few constraints on the solution so it can take an enormous number of different forms and still be valid. There isn't gravity, friction or the various other things that constrain physical solutions.
And each of those almost limitless solutions has different trade-offs related to a large number of different variables. It takes many many years of seeing the different impacts of those design decisions across the different variables within the constraints of different domains and contexts.
It's truly a combinatorial problem with so many dimensions that no human is smart enough to just simulate the impact of design decisions in new and unfamiliar domains+context.
You could argue that there could be standard ways of doing things within specific domains+context, but the context is so varied between companies, and business priorities have so many differences that I don't think it's a solvable problem.
The best we can do is have higher level patterns and approaches to specific types of problems that can guide people based on others experiences, but all of that still needs to be mixed and matched for the specific problem space.
I agree with your assessment and didn't say it's easy. If it was then we'd already have solved it.
But that doesn't mean it can't be solved. I'm sure 150 years ago people said something similar about flight. Not a great analogy, but the point stands that if we keep working on it and have a goal then in a few decades we could get closer to actual engineering than the mess we are in today.
For me, software design is more comparable to business system and organization design. It’s just that instead of humans executing the processes you design, it’s machines doing it.
I don’t think business system and organization design is approached like bridge design either, is it?
Also, bridges, houses and surgeons can physically kill people if something goes wrong. Software that can physically kill people, such as that in airplanes or missiles, is actually treated quite differently from most software, I think? I don’t have experience in those industries those so I can’t comment on the specifics of how is it different, but my impression is that things are a lot more rigid. Business organization design also can’t directly kill people.
In general, I think that there is a fundamental tension between looseness and flexibility of operations and innovation. If you are super rigorous and have set in stone best practices, it is going to be harder to find new ways of doing things that work better.
I’m not really bothered if people don’t consider software to be a serious engineering discipline. I’m not sure I do either. If someone wants that kind of thing, I’d recommend they go into a different engineering discipline, rather than trying to make software like that.
Tell me you don't know the first thing about engineering without telling me...
This is the engineering process. If you put five engineers of any discipline in a room, you will get five different answers. Every contractor and architect has their own ideas about how things should be done.
Furthermore, we do build houses this way, even in the modern age with building codes. The builder is going to do whatever they can get away with and this is a universal truth. The only reason bridges are held to such high standards is because of the monetary cost of a collapse.
The "norms" are not what you think they are. They're tradition, they're "we've always done it this way". What you're talking about are laws written for public safety. Those laws only exist because we tried and failed to do things a certain way and the cost in money or lives was untenable.
> "I recognize a good solution when I see it" is just not good enough for a serious engineering discipline.
You're conflating engineering with science. A scientist will rigorously test and validate, but a huge part of engineering is just recognizing a good solution. Of course there's a lot of testing and validation in engineering as well, but not with the kind of rigor you're implying.