Let’s say you have an incident, and you need a resolution. Do you send out the AI for the hunt and fix it within 5 minutes, or spend 30 minutes going around, figuring out the problem yourself?
I’m on the camp of using agents as much as you can, but make sure to understand what they did. However, it will still atrophy your skills, no matter what. It’s just different times now, as on the job you’re expected to deliver faster. And obviously that means less time spending on your craft, honing your skills and etc.
Digging a well using manual tools is a skill. Using a machine to dig is also a skill. They are distinct skills. So if people change the way work is done, skills themselves shift.
Using AI appropriately and efficiently and safely is a skill. Similar to using a mechanized excavator. It can do the work 10x faster but it can also do a lot of damage quickly as well.
Those who don't realize this and not adapt to the new skill set required by the industry will lose their jobs. It will cause a loss/atrophy of prior skills and development of new skills but that is inevitable.
Those who have the prior set of skills do have an advantage over those who don't have them. But the new set of required skills can be developed without having the earlier ones too. The most proficient in the new set of skills may or may not be the ones that were most proficient in the old skills.
> Let’s say you have an incident, and you need a resolution. Do you send out the AI for the hunt and fix it within 5 minutes, or spend 30 minutes going around, figuring out the problem yourself?
Let’s say you have an incident, and you need a resolution. Do you spend 5 minutes tinking, then solving the problem or send out the AI for the hunt and spend 30 minutes grappling with hallucinations and non-sensical output?
The 5 minutes I’ve seen is usually by someone that will send the complete output to someone else to decipher, while saying they have figured it out. Then they’ll turn into a meat proxy between the team and the agent.
In real life, for systems where incidents that require such rapid responses, you usually have fail-safes an killswitches. When you were designing your system, did you put in place alerts and monitoring to mitigate risks?
I’m not sure if you’re running the latest frontier models. Every single engineer (and myself included!) has mentioned numerous times how nothing beats AI in debugging. My other guess would be that you have non-standard set up that Opus 5.5 doesn’t know off the bat how to debug.
I wouldn’t say the same a year ago, but usually we get preliminary reports during incidents before someone can even auth to debug. Almost all the time, nowadays, the report is correct. We obviously verify it.
"Latest frontier models" seems a bit like acting like smarter reasoning can fill in context that doesn't exist in various systems without guessing. It's convenient because it's always the users fault for not working in the platonic ideal observability setup that can feed everything on a plate. And if you're in that situation how much is the agent adding over incremental improvements?
It can't be both "so easy and hands free anyone can use it" and also "well they must be using the wrong model/holding it wrong if it's not working for them". If there weren't tradeoffs of the different models then it wouldn't be exposed to the user.
> Let’s say you have an incident, and you need a resolution. Do you send out the AI for the hunt and fix it within 5 minutes, or spend 30 minutes going around, figuring out the problem yourself?
It is far more often the case that you will spend more time getting the answer out of the LLM than you would've gotten yourself. They suck ass at troubleshooting systems, very frequently jumping to completely implausible scenarios that no human would think of as the first thing to try, and almost never asking questions to clarify the situation the way a human does.
This has not at all been the case for me. LLMs (even the more-capable local models) are exceedingly good at troubleshooting when given proper context (clues, hints, a decent system prompt/skill that provides even the barest hint of how things work)
What you're describing sounds more like what would happen if you gave a chatbot without access to the repo as little context as possible and asked it to come up with a solution, blind.
> They suck ass at troubleshooting systems
Given the right tools and instructions I find them to be consistently superhuman. It's unreal.
I have an automated system responding to issues occurring on oceanconnect.ca, and every single diagnosis has been correct for months. Of course I still verify, I tune solutions, I review everything. But it's right. It's not a trivial system. It has a fairly large frontend, a large-scale model processing pipeline on the backend, a fairly complex database, and a queue for getting new data into the system. Claude does a stellar job of navigating and understanding problems, even when they're not 'real' or as they initially appear to be; things like quirks in old versions of Safari, or the model pipeline fails due to data integrity issues from the providers.
I've rarely encountered cases where I need to steer a frontier LLM much when troubleshooting. They're exceptional.
Sorry, this might be a thing in your org, and I have no idea hoe I can convince you, but it’s just not true in our environments. Maybe you might have a very non-standard set up, but frontiers do amazing job.
I’m a troubleshooter and I’ve managed human troubleshooters. The LLMs are better - if they are grounded in either IAC or api access to the infra.