Author must not have worked in enterprise software before.
That's a classic trick where the developer will push back on the bug author and say "I can't reproduce this, can you verify it with the latest version?" without actually doing anything. And if it doesn't get confirmed then they can close it as User Error or Not Reproducible.
Of course, the only way to counter this is by saying "Yes I verified it" without actually verifying it.
From experience with Microsoft (paid) support (after doing 5 tickets because it's never the right team and apparently moving tickets internally is for losers), they will ask for proof of the reproduction. And they will take every opportunity to shift the blame ("Oh I can see in the log you're running an antivirus, open a ticket with them. Closed").
I recompiled OpenSSL to make s_server -www return the correct, static XML blob for a .NET application that was buggy to make a reproducer for them that didn't rely on our product at all and which could be self-contained on a very barren windows VM they could play with to their heart's content and which didn't even care about the network because everything was connecting via loopback, so they couldn't blame that, eitehr.
Turns out there was a known bug in Microsoft schannel that had yet to be patched and they'd wasted weeks of our effort by not searching their own bug tracker properly.
Searching bug trackers has always been problematic unless you have an exact error message or code. Fortunately vendors like Linear are now embedding AI/LLM technology into their products, which helps a lot with fuzzy searching and with retroactively finding duplicate bug reports.
My favourite variant of this merrygoround is when they ask you to demonstrate the issue live in a Teams session, you do so, and there's this moment of silence followed by an "Oh... I see".
Then you assume, naively, that this means that they've recognised that there really is a product problem and will go off and fix it. However, then in turn the support tech needs to reproduce the the issue to the development team.
They invariably fail to do so for any number of reasons, such as: This only happens in my region, not others. Or the support tech's lab environment doesn't actually allow them to spin up the high-spec thing that's broken. Or whatever.
Then the ticket gets rejected with "can't reproduce" after you've reproduced the issue, with a recorded video and everything as evidence.
If you then navigate that gauntlet, the ticket is most typically rejected with "It is broken like that by design, closed."
It'd kind of sad, how the market went. I suppose there are pluses too.
But back in the 80s and 90s, margins were significantly higher. If you look at hardware, I recall selling hardware with 30% margin, if not more... even 80% on some items.
Yet what came with that was support, support, support. And when you sell 5 computers a month, instead of 500, well.. you need that margin to even have a store. Which you need, because no wide-scale internet.
On the software side, it was sort of the same. I remember paying $80 for some pieces of software, which would be like $200 today. You'd pay $1 on an app store for such software, but I'd also call the author if there was a bug. He'd send an update in the mail.
I guess my point is, in those days, it was fun to fix issues. The focus was more specific, there was time to ply the trade, to enjoy it, to have performant, elegant fixes.
Now, it's all "my boss is hassling me and another bug will somehow mean I have to work harder", which is .. well, sad.
High end enterprise products still come with support. That's literally what customers are paying for: a single throat to choke.
Exactly! The "pay a lot of money but get really good support" tier still exists just about everywhere. You just didn't do the first part.
That kind of attitude disgusts me. Like it's someone else's job to have a sense of accountability. They would not remain employed in my company.
When I developed software I would jump right on top of any bug reports immediately, and work until they were fixed. I was grateful to my customers for bringing them to my attention.
It is different when you have a billion customers, all with different setups. At that scale, you notice real defects through product telemetry, support ticket volume, or trusted channels. You receive a high volume of bug reports that are due to user confusion, misconfiguration, or misbehavior of other software on the device - where solving an issue for one customer doesn't result in improvements for the other billion. Triage, filtering, and winnowing are necessary here.
My guess it's just the emergent behavior that results when a company doesn't provide developers time to fix bugs.
If their week is already booked full just trying to keep up with the roadmap deadlines, a bug ticket feels like being tossed a 25lb weight when you're drowning.
You could say: "but have pride in your work!"
But if your company only values shipping, not fixing, that attitude doesn't make it through the first performance review.
What I've found to be most effective for program management is to set aside a maintenance team separate from the feature teams. The roadmap is then planned without counting anything for the maintenance team and they deal with bug tickets as they come in. Rotate the assignment periodically so that every developer has to occasionally spend a few months on the maintenance team.
Yep. On the other side of the curtain this often isn't nefarious. It's a simple cost/benefit analysis of spending time on something that one user is complaining about versus a backlog of higher business priorities. I've seen this in my work and it makes me sad for the user, but it often does take a bit of effort to spear these bug reports through.
I totally understand that from the perspective of individual employees: they have little incentive to do more than the bare minimum to close tickets. But this behavior is typically a symptom of broken corporate culture and failure to align internal metrics. For every customer who takes the trouble to submit a formal bug report there are likely many others who just live with it, and badmouth you to other customers. Doing deep investigations of even minor bug reports also tends to expose other, more serious latent bugs. And root cause analysis allows you to create closed-loop solutions to prevent similar future bugs.
Large monopolistic tech companies like Apple and Microsoft can afford to ignore this stuff for years because there are few realistic alternatives. But longer term eventually a disruptive competitor comes along who takes product quality and customer service more seriously.
Don't just a bug report by its cover, judge it by its merits. We're all biased to dismiss them and find an excuse to ignore them. But that just leads to bad software.
https://m.slashdot.org/story/391921
There's also going to be mountains of bugs resulting from cosmic rays hitting the computer, defective ram chips, weird modifications of the system the reporter hasn't mentioned.
You could sink an infinite amount of time investigating and find nothing. At some point you have to cut off the time investment when only one person has reported it and no devs have been able to reproduce it.
If bug contains instructions for reproduction most of that will be eliminated.
There's obviously some nuance here, but the fact is that much modern software is riddled with bugs, and this is sub-optimal for everyone (both software users and software builders). Most of the bugs which frustrate and irritate software users are not due to uncontrollable events such as cosmic rays flipping a bit. Most of them are plain old code defects.
But, you do have a valid point. Allow me to rephrase it this way: The answer is not for software companies to spend unbounded amounts of engineer time chasing every reported bug.
But there are ways that we, as an industry, can do better, and it's not by pouring all our time into chasing hard-to-diagnose bugs. Here are a few ways that I personally see:
1. Some very powerful technologies for finding many bugs with little engineering effort already exist, but are not widely used. As an example, coverage-guided fuzzing is amazingly good at finding all kinds of obscure bugs. The idea of coverage-guided fuzzing was known from the 1990's, but it took AFL (in ~2013) to take it mainstream. Even now, much of the industry is not benefiting from the awesome power of coverage-guided fuzzing. And there are other, equally powerful techniques which have been known for a long time, but are even less accessible to most software developers.
So: spread the word about such techniques, and for programming language/platform developers, work on making them more easily applicable. This could help many software companies to catch a great number of bugs before they ever go to production.
2. Similarly, there are extant historical computing systems which had very powerful debugging facilities, much better than what is currently available to most developers. The ideas on how to make our platforms more debuggable are already out there; it's now a matter of popularizing those ideas and making them readily accessible and applicable.
3. Since it's widely known that many bugs (real bugs, not "cosmic rays") are extremely hard to reproduce, an admirable target for us to aim for as developers is to implement debug logging in a way which allows us to root-cause most obscure bugs just by examining the logs (i.e. no need to search for a reproducer). Some real-world systems have achieved that goal, with very good results.
4. While there is currently much buzz about using LLM-based coding agents to write code, I think an almost better use case for coding agents is in triaging bug reports, diagnosing the bugs, finding reproducers, etc.
I've recently had a couple of shocking experiences where, just given a written description of an intermittent, hard-to-diagnose bug, a coding agent was able to search an entire codebase, identify the exact cause, and write a reproducer test case. (And this after multiple experienced human programmers had looked at the issue but failed to identify the cause.)
In summary, I think there are ways to "cut the Gordian knot" of bug reports.
What if no devs even tried to reproduce it, and they have no reason to believe they've fixed the bug with any other changes?
That seems to be the case described in the article. In such a situation, I think it's dishonest to ask the reporter to expend even more effort when you've spent zero. Just close it if you don't want to do it, you don't have to be a jerk to your customers, too, by sending them off on a wild goose chase.
Otherwise, why not ask the reporter to reproduce the issue every single day until you choose to fix it in some unknown point in the future, and if they miss a day, it gets closed? That seems just as arbitrary.
I'll steal this to my projects bug template! /s
"Please consider cosmic rays hitting the computer, defective ram chips, weird modifications of the system before submitting the bug. Unlesss you explicitly acknowledge that, your bug will be closed automatically in 30 days. Thank you very much"
Yeah competition works. I don't like nexus that much but they accept every ticket I've opened and fix it the next release. Two things I think affect that. One, my ticket has the name of a fortune 100 next to it. Two, artifactory will eat them alive if they don't keep customers happy.
If you're having trouble reproducing, tag "needs verification" or something else. But closing a ticket isn't triaging, it is sweeping problems under the rug
It's a false dichotomy - something being "a simple cost/benefit analysis" doesn't remove the ethical dimension, and can absolutely be nefarious. A movie villain saying "it was just business" doesn't make their actions less villainous.
I’d argue that there should be no higher business priority than shipping a product you already sold. If you sold a product and your customer spends their time documenting exactly why and how you sold them something that’s broken, you should make that a high priority. As a natural progression, you’ll start shipping less buggy / better tested products and that’s how you unlock yourself from the obligation you made to your existing customers to do other work.
Not directed at you of course, just the proverbial “you” from the frustration of a purchaser of software.
Careful saying that too loudly, the “ship new features at all costs” gang will come for your head. They don’t approve of things like “quality software” and “making stuff that works past the demo and cursory inspection” or “actual user utility”.
I can sort of back that for desktop apps but telemetry is so trivial for webapps needing a reproducer is almost an embarrassing admission the operator has no clue what they're doing.
Error tracking and tracing make it fairly straight forward to retroactively troubleshoot unreproducible issues.
> Author must not have worked in enterprise software before.
Or with open source projects. Fucking stalebot.
As an open source maintainer, I feel that statement is really unfair. Yes, we do sometimes close bug reports without evidence they are fixed. But:
- We owe you nothing! And the fact that people still expect maintainers to work for them is really sad, IMHO.
- Unlike corporate workers, nobody is measuring our productivity therefore we have no incentive to close issues if we believe they are unfixed. That means that when we close the issue, we believe it has a high chance of being fixed, and also we weigh the cost of having many maybe-fixed open issues against maybe closing a standing issue, and (try to) choose what's best for the project.
It's not about expectation of work (well, there's some entitled people sure.)
It's about throwing away the effort the reporter put into filing the issue. Stale bots disincentivise good quality issues, makes them less discoverable, and creates the burden of having to collate discussion across N previously raised issues about the same thing.
Bug reports and FRs are also a form of work. They might have a selfish motive, but they're still raised with the intention of enriching the software in some way.
> That means that when we close the issue, we believe it has a high chance of being fixed
I agree with this iff it's being done manually after reading the issue. stalebot is indiscriminate and as far as "owing" the user, that's fair, but I'd assume that the person reporting the bug is also doing you a favor by helping you make things more stable and contributing to your repo/tool's community.
I partially agree, but even with stalebots nobody is measuring the maintainers' productivity. So when they made the choice to use stalebots, they did that because they believe that's best for the project. It's different from corporate.
Nobody is measuring their productivity, but people definitely look at how many open issues they have and potentially how long those issues have existed. They’re likely incentivized to close issues for appearances.
Why do you close the issue then?
Because I have a reason to believe it's fixed, I have many more like it and it's difficult to reproduce. Simple :)
Because open source is corporate now
Or even non-software tickets at large corporations. I reported a water dispenser filling too slowly at my office because it took me a few tries just to fill my 1L water bottle. They said it was fixed and closed it.
It was not fixed. So I took a video of myself refilling my water bottle, attached it to the ticket, and re-opened it. They actually fixed it after that. The video was 2m12s long (and I spent god knows how long making the video file small enough to attach to the ticket lol)
this is actually a good example of how a more detailed issue will have a higher chance to be addressed. I don't know what information that's your previous report is lacking, but the video certainly give more information that the maintainer can pinpoint the cause and act on it. The ability to pinpoint the cause from the report is a godsent for maintainers, it drastically reduce the time to investigate the cause, thus able to act immediately.
Some of the information in this can may be:
* how "slow" exactly the process is related with normal behavior. If it's just said "slow" on previous report, it's easy to be dismissed
* the dispenser's behavior, such as if the water flow is consistently low volume or clogged intermittently, or if the dispenser is struggling to fetch from water source, etc
Take a look at Anthropic's repo. They auto-close issues after just a few weeks.
I don't think I've seen an issue of theirs that wasn't auto-closed.
Wait, isn’t software engineering a solved problem?
Yes, that’s why they have such great up time. They don’t go down multiple times per day.
Yes
Fuck stalebot.
All my homies hate stalebot
I got reeeeally good at producing repro gifs that I could plug straight inline into email replies to "can't repro"; it's forever clear that most developers either don't know how to test the product they are building, or simply can't be bothered to try.
Hi, bigcorp employee getting showered with tickets here.
I don't have enough time in the day to deal with the tickets where the reporter actually tries, let alone the tickets where they don't.
If I tell you to update your shit, it's because it's wildly out of date, to the point that your configuration is impossible for me to reproduce without fucking up my setup to the point that I can't repro 8 other tickets.
Back when I worked at Apple I would just try it in whatever I had installed. If it didn't reproduce I'd write "Cannot reproduce in 10.x.x" and close it. Maybe a third were like that, duplicates of some other issue that was resolved long ago.
Anyone that attached a repro file to their issue got attention because it was easy enough to test. Sometimes crash traces got attention, I'd open the code and check out what it was. If it was like a top 15 crash trace then I'd spend a lot longer on it.
If the ticket was long and involved like "make an iMovie and tween it in just such and such a way" then probably I'd fiddle around for 10-15 minutes before downgrading its priority and hope a repro file would come about.
There were a bunch of bug reports for a deprecated codec that I closed and one guy angrily replied that I couldn't just close issues I didn't want to fix!
Guess what buddy, nobody's ever going to fix it.
The oldest bug like that I ever fixed was a QuickDraw bug that was originally written when I was 8 years old but it was just an easy bounds check one liner.
But the mistake OP is making is assuming this one thing that annoyed him somehow applies to the whole Apple org. Most issues were up to engineers and project managers to prioritize, every team had their own process when I was there.
> But the mistake OP is making is assuming this one thing that annoyed him somehow applies to the whole Apple org. Most issues were up to engineers and project managers to prioritize, every team had their own process when I was there.
Except this same shit keeps happening with multiple teams.
Judging from your mention of QuickDraw, which was removed entirely from macOS in 2012, perhaps your Apple experience is now out of date.
Nah, you're just making shit up.
What specifically do you claim I'm making up?
That the ~50000 engineers at Apple are conspiring to close your tickets in the exact same way. It's ridiculous
Not my tickets specifically. I don't think they're out to get me individually. On the contrary, this is a common practice, which affects many developers. I just happen to be relatively loud, as far as blogging is concerned.
Yes I understand that. ~50000 engineers aren't conspiring to close all tickets that way. It's a stupid line of thinking.
More than likely your steps to reproduce are too laborious to receive attention relative to the value fixing the bug would provide. That's why they're asking you to verify it still happens. Seems pretty simple right?
There's also a strong chance your ticket was linked as a duplicate of some other issue that was fixed in the beta and they want you to verify that's the case but they won't expose their internal issue to you for a variety of reasons.
> ~50000 engineers aren't conspiring to close all tickets that way.
I didn't say that either. It's happened to me only sporadically, but multiple times.
I agree with you that teams within Apple manage their own tickets. Perhaps some individual teams are declaring bug bankruptcy at some point, so only their bugs would go out for verification. I don't really know. I wish I did. What I do know is that multiple teams have done this at different points.
There's indisputably a company-wide DevBugs canned response for this. It's the same exact language every time. You can even Google it.
> It's a stupid line of thinking.
Please respect the HN guidelines: https://news.ycombinator.com/newsguidelines.html
> More than likely your steps to reproduce are too laborious to receive attention. That's why they're asking you to do it.
It was much more laborious for me, because I do not install the macOS betas.
> Seems pretty simple right?
No, it doesn't explain why specifically, after 3 years, they were suddenly asking me to verify with macOS 26.4 beta 4.
Yes that's a thing, but never with external customers in public betas
I think that's entirely dependent on the workload the company is placing on their support staff. If Apple decides the techs should be handling 10 tickets at once, then the techs have a choice:
1. Tell everyone to update their shit, and close tickets if they don't.
2. Waste several hours per day uninstalling and reinstalling 10 versions of the same program.
One of these will allow you to close lots of tickets immediately, and handle the remaining ones as efficiently as possible. Yay! Good job, peon! You get a raise!
The other approach will result in a deep backlog, slow turnaround times, and lower apparent output from management's perspective. Boo! Bad job, peon! You're fired!
Please tell us where you work so we can avoid all of your company’s software. Unless it’s Microsoft, because we’ve already seen the results of that attitude there.
I don't see how it's an unreasonable request. If you demand that I work with some ancient version, I then have to install and uninstall said program every time I work on your ticket specifically. You will be prioritized last, because my effectiveness is measured by how many tickets I close.
just bc everyone is calling you insane: you are being extremely reasonable.
> If you demand that I work with some ancient version, I then have to install and uninstall said program every time I work on your ticket specifically.
You completely missed the point of the blog post. Apple was in the process of developing macOS 26.4 beta 4, and they wanted me to install the beta just to "verify" the bug.
Apple could test my bug with 26.4 beta 4 a heck of a lot easier than I could. Nobody was asking Apple to install some ancient version.
> my effectiveness is measured by how many tickets I close.
That was one of the points of the blog post: this is a perverse incentive from management.
Note what you did not say: "my effectiveness is measured by how many bugs I fix." So engineers are incentivized to close tickets even if the bugs they report are unfixed. This is how a company ends up with crappy, buggy software.
I'm with you on the Apple thing, that's asinine.
The parent comment is talking about the broader practice of people telling you to update and then repro again. That's a completely legitimate thing to ask, given both the perverse corporate incentives and the basic reality that version toggling makes a tech far less efficient at solving all tickets, not just yours.
[flagged]
[flagged]
I've worked with enterprise software. The result its that people will eventually just wait a few hours/days and lie if they even care enough to do that. The perverse incentives destroy what utility a bug tracker could bring. Int theory transparency could help by changing the incentives if third parties analyze the metrics and call out bullshit to an audicence that matters.
> Of course, the only way to counter this is by saying "Yes I verified it" without actually verifying it.
I'm not going to lie. That's not who I am. If Apple really wants to close a bug report when the bug isn't fixed, that's on their conscience, if they have one.
Companies have no consciences.
Companies are made of humans who do.
If you are a veteran of software in a big company, we all know there will be weekly or bi-weekly meetings that some PM will set up. All the PM will do is go over the JIRA tickets and be like "is this still happening". Default answer is "no", as in "I didn't even try to reproduce it, do you think I have time to even do it?". Default answer by spineless QA person is also "didn't try it again yet". Then, the PM closes the ticket. It is much easier for QA person to say "Yes I verified it" if you are remote and developer cannot see the lies on your bad poker face.
Ooh this gives me an interesting passive-aggressive idea to counter pointless "is this still relevant" questions. "No, I haven't hit this in the last 2 days." "No, I haven't hit this since I gave up trying to do it with your tool." And so forth.
The less passive-aggressive version is to use this obviously-unhelpful answer of the obviously-unhelpful question, to actually have a conversation to get the PM to recognize that the default state of a ticket is in fact "no change." Ultimately that may turn into a stale bot if the PM realizes the policy they actually want is some sort of timeout, but at least it's not a time consuming meeting!
(Note, a cathartic thought experiment, but not really good manners to actually do!)
Absolutely spot on LOL
I also hate this pressure of it being on the user to come up with a minimal reproducing example. That means that any bug of any moderate complexity will never get fixed because you can't always reduce them to a few steps and they may be statistical.
A bug is a bug, no matter the developers' opinion or the complexity of the bug.
[dead]