What I thought was fascinating, and should be a warning sign to everyone here:
Before beginning the study, the average developer expected about a 20% productivity boost.
After ending the study, the average developer (potentially: you) believed they actually were 20% more productive.
In reality, they were 0% more productive at best, and 40% less productive at worst.
Think about what it would be like to be that developer; off by 60% about your own output.
If you can't even gauge your own output without being 40% off on average, 60% off at worst; be cautious about strong opinions on anything in life. Especially politically.
Edit 1: Also consider, quite terrifyingly, if said developers were in an online group, together, like... here. The one developer who said she thought it made everyone slower (the truth in this particular case), would be unanimously considered an idiot, downvoted to the full -4, even with the benefit of hindsight.
Edit 2: I suppose this goes to show, that even on Hacker News, where there are relatively high-IQ and self-aware individuals present... 95% of the crowd can still possibly be wildly delusional. Stick to your gut, regardless of the crowd, and regardless of who is in it.
> Also consider, quite terrifyingly, if said developers were in an online group, together, like... here. The one developer who said she thought it made everyone slower (the truth in this particular case), would be unanimously considered an idiot, downvoted to the full -4, even with the benefit of hindsight
Yeah, this is me at my job right now. Every time I express even the mildest skepticism about the value of our Cursor subscription, I'm getting follow up conversations basically telling me to shut up about it
It's been very demoralizing. You're not allowed to question the Emperor's new clothes
This should really be top comment. The problem is this tools can really give us some value in certain type of areas, but they are not like they are marketed.
Given how deadlines/timelines tend to (not) work in SWE, this is not surprising.
Perhaps; but this is a developer's own output with an AI tool, compared against their own historical output when they didn't use it. Apparently, the average developer (read: quite possibly most people here) can't even hit the broadside of a barn in estimating their own productivity.
That this is generally a problem, and was established as such before software development existed (the big thing people usually point to is a RAND corp from the 1940s) and is the whole motivation for Wideband Delphi estimation methods invented shortly afterwards (of which agile "planning poker" is simply a particular more recent realization) for forward estimation, and why lean methods center on using a plan-do-check-act cycle for process improvements rather than seat of the pants and subjective feel.
But despite the popularity of some of this (planning poker, particularly; PDCA for process improvements is sadly less popular) as ritual, those elements have become part of a cargo cult where almost no one remembers why we do it.
But this is still regarding forward estimating of future work, whereas GP is talking about gauging actual, past work done. The problems with forward estimation are indeed widely known, but I doubt most people realize that they are so bad at even knowing how productive they were.
That doesn't surprise me at all. Isn't software engineering in essence about being constantly confronted with new problems to solve and having to come up with a sufficient one on the fly? It seems very hard to estimate this, even if you know yourself well.
They were 20% underestimating how long it took them to do a 1-8 hr task that they had just completed.
It's like Tog's study that people think Keyboard is faster than the mouse even when they are faster with the mouse. Because they are measuring how they feel, not what is actually happening.
https://www.asktog.com/TOI/toi06KeyboardVMouse1.html
That is a very weird set of findings.
This one in particular:
> It takes two seconds to decide upon which special-function key to press.
seems to indicate the study was done on people with no familiarity at all with the software they were testing.
Either way, I don't think there is any evidence out there supporting that either of keyboard-only or mouse-only is faster or equivalent to keyboard+mouse for well known GUIs.