I’ve conducted about 60 interviews this year, and have spotted a lot of AI usage.
At first I was quite concerned, then I realized that in nearly all cases I’d spotted usage, a pattern stood out.
Of the folks I spotted, all spoke far too clearly and linearly when it came to problem solving. No self doubt, no suggestion of different approaches and appearance of thought, just a clear A->B solution. Then, because they often didn’t ask any requirements questions beyond what I initially asked, the solution would be inadequate.
The opinion I came to is that even in the best Pre-AI era interviews I conducted, most engineers contemplate ideas, change their mind, ask clarifying questions. Folks mindlessly using AI don’t do this and instead just treat me as the prompt input and repeat it back. Regardless of if they were using AI or not, I won’t know ultimately, they still fail to meet my bar.
Sure, some more clever folks will mix or limit their LLM usage and get past me, but oh well.
I interviewed a guy in person and he paused for 5 seconds, then wrote a perfect solution. I tried making the problem more and more complicated and he nailed it anyway, also after a brief pause. We were done in half the time.
Maybe he just memorized the solution, I don’t know.
Would you fail that guy?
It depends, I had some interviews like this that I suspected. For context, most of the interviews I conduct are technical design related where we have a discussion, less coding. So in those it is quite open ended where we will go, and there are many reasonable solutions.
In those cases where I’ve seen that level of performance, there have been (one or more of):
- Audio/video glitches.
- candidate pausing frequently after each question, no words, then sudden clarity and fluency on the problem.
- candidate often suggests multiple specific ideas/points to each question I ask.
- I can often see their eyes reading back and forth (note; if you use AI in an interview, maybe dont use a 4K webcam).
- way too much specificity when I didn’t ask for it. For example, the topic of profiling a go application came up, and the candidate suggested we use go tool pprof and suggested a few specific arguments that weren’t relevant, later I found in the documentation the same exact example commands verbatim.
In all, the impression I come away with in those types of interviews is that they performed “too well” in an uncanny way.
I worked for AWS for a long time and did a couple hundred interviews there, the best candidates I interviewed were distinctly different in how they solved problems, how they communicated, in ways that reading from an llm response can’t resemble.
The point is that I interviewed the guy in person and he nailed it 200%. If you interviewed him online you would likely come to conclusion he’s a fake per the criteria you specified, wouldn’t you?
It’s not a rubric I’m checking off for interviews. And in person it’s more straightforward to assess a candidate than questioning if they are using any aids over video… whats your point?
He made the point clearly, stop dodging the question...
Wasn’t trying to dodge, I misunderstood the premise.
If this was in person, then no I likely wouldn’t fail them. However, In all my in person interviews I’ve conducted, I’ve never seen that even from the best candidates, that’s why I also find it odd over video.
I might hire him, but I would insist he clock out for his 5 second paused. We can’t have him wasting company time like that.
you pay devs hourly?
Apparently by the second. Don't blink too often.
I’m running a high precision outfit over here ya know
Only Type A run through walls folk
The real problem will be in 5 years, when current university students having their brains melted by AI that somehow luck into entry level positions can’t ever get to senior level because they’re too reliant on AI and they literally don’t know how to think for themselves. There will never again be as many senior engineers as there are today. There won’t be any good engineers left to hire.
Look around you. 15 years ago we didn’t have phones and now kids are so addicted to them they’re giving themselves anxiety and depression. Not just kids, but kids have it the worst. You know it’s gonna be even worse with AI.
Most departments at companies run on zero to two good engineers anyway. The rest are personality and nepotism hires limping along some half-baked project or sustainment effort.
Most people in my engineering program didn’t deserve their engineering degrees. Where do you think all these people go? Most of them get engineering jobs.
I’m gonna assume you’re being facetious here. I’ve been in tech for 15 years and I’ve never met a “nepotism hire”. Most of my coworkers have been incredible people.
But in case you’re serious, there’s an old saying that says if everywhere you go smells like shit maybe it’s time to check your shoes.
I don’t work in tech.
Why are you commenting on a post about technical interviews for engineers then
Jumping straight to the optimal solution may also indicate that candidate have seen the problem before.
The funny thing is, they don’t. They often jump to a solution that lacks in many ways, because it barely addresses the few inputs I gave (since they asked no follow up, even when I suggest they ask for more requirements).
Can I ask - out of the 60 interviews, roughly how many times did you suspect AI usage?
Probably about 10 or so.
> most engineers contemplate ideas, change their mind, ask clarifying questions
I don't disagree at all. I find it slightly funny that in my experience interviewing for FAANG and YC startups, the signs you mentioned would be seen as "red flags". And that's not just my assumption, when I asked for feedback on the interview, I have multiple times received feedback along the lines of "candidate showed hesitation and indecision with their choice of solution".
I work for a FAANG, have done interview training and numerous interviews. We are explicitly trained that candidates should be asking questions, second guess themselves etc.
Hotshot FAANG and YC startups don't want humans, they want zipheads[0].
[0] https://www.urbandictionary.com/define.php?term=Ziphead
Yeah that is definitely something that is subject to the interviewers opinion and maybe company culture. To me, question asking is a great thing, though the candidate eventually needs to start solving.