You could filter then out much more effectively by letting them sit in a room by themself to write the code, that way you aren't missing out on good candidates who can't function when under abnormal stress(that has nothing in common with the actual job).
I've had take home problems for job interviews that were given a few days before and during the actual interview I only had to explain my code. But I wouldn't be sure this still works as a useful candidate filter today, given how much coding agents have advanced. In fact, if you were a sr dev and had a bunch of guys to bounce this problem back and forth, it wouldn't even have filtered out the bad ones back in the old days. There is very little that is more telling than seeing a person work out a problem live, even if that sucks for smart people who can't handle stress.
I have found over the years that I learn more by asking easier questions and interacting with candidates as they think through problems out loud. Little things that test the critical ability to craft a Boolean expression to accurately characterize a situation can be explored in a brief interview where you have some assurance that they're working on their own, and not just getting an answer online or from a smart roommate. (Sample: given two intervals [a,b] and [c,d], write an expression that determines whether they intersect.). Candidates that have lots of trouble programming "in the small" are going to have trouble programming in the large as well.
What I find effective (on both sides of the interview table) is not only asking easier questions, but active encouragement to first talk out and work through the most fundamental, basic aspects of the problem and it's simplest solutions before moving into more advanced stuff.
I think a lot of experienced people's brains lock up in an interview when asked simple questions because they assume that they're expected to skip straight past the "obvious" solution and go straight for some crazy algorithm or explain the fine points of memory/time tradeoff. This often doesn't present as intended - it looks to the interviewer like you don't even know the basics and are grasping at straws trying to sound smart.
If people can explain their decisions, I'd say it's fair game. It would be nice to know up front if someone used AI of course.
The other implication here is that if a candidate can use AI for a take home and ace the interview, then maybe the company doesn't have as tough of problems as it thought and it could fill this seat quickly. Not a bad problem to have.
Leetcode for CRUD app positions is overkill.
I don’t use those LLM tools, but if someone can pass the test with LLM tools, then they can pass the test unless there’s something special about the environment that precludes the LLM tools they use.
Even before the coding assistants I was not happy with homework. It was not discriminatory, everyone looked the same, https://andrewpwheeler.com/2022/03/24/i-have-no-clue-how-to-....
> But I wouldn't be sure this still works as a useful candidate filter today, given how much coding agents have advanced.
Prior to ChatGPT coming out, I gave a take home test to sort roman numerals.
What before was a "here's a take home that you can do in an hour and I can check the 'did you write the code reasonably?'" is now a 30 seconds in ChatGPT with comments embedded in it that would make an "explain how this function works" to be less useful. https://chatgpt.com/share/688cd543-e9f0-8011-bb79-bd7ac73b3f...
When next there's an interview for a programmer, strongly suggest that it be in person with a whiteboard instead to mitigate the risks of North Korean IT workers and developers who are reliant on an LLM for each task.
My solution for this was a propose a problem obscure enough that no LLM tool really knows how to deal with it. This involved some old Fortran code and obscure Fortran file format.
How would someone explain code that was vibe-coded?
You can have the AI explain it to you. There's also a middle ground between vibe coding and "I can code some things but never could have coded this without an AI".
Doesn't even have to be AI. Give me some random file from the Linux kernel and I could probably explain everything to you if you gave me a few hours. But that doesn't mean I would ever be able to write that code.
I don’t disagree, but in those interviews the explanation is also a bit of a q&a, so an effective interviewer can detect candidates who only memorized things. Someone who can ace a q&a about Linux code is already better than average.
Isn't that a good thing? The fact that the candidate dumped out code that they didn't write is often called "cheating". The fact that candidates can't explain it (because they didn't write it) means it's a good test of something most interviewers find unacceptable.
Are you asking how they would get that info they didn't have / couldn't come up with? Because you can literally have a chatbot explain every line to you and why it is there and even ask it about things you don't know like a teacher. And for simple problems this will probably work better than with actual humans.
I assume questions to explain the code would be extremely specific about why you did something on a specific line, or why you chose to design this part that way instead of another, to detect plagiarism and vibe coding, not a request for a prepared monologue.
We live in a remote world where a hiring process like this is less of an option in most cases
Leaving aside that many companies have pulled back from remote to at least some degree, I'd always push for an in-person day for a variety of reasons. In general, the cost is nothing for a late-stage/end-stage confirmation. And, honestly, a candidate that just doesn't want to do that is a red flag.
> In general, the cost is nothing for a late-stage/end-stage confirmation.
One in-person day costs a nearby candidate about 3 days, and a more remote candidate anything from a week to a couple of months depending mostly on where you are. And yeah, it also costs some money that maybe you will reimburse.
It doesn't cost you much, that's for sure. But if it's for a full-remote position, it's absolutely not a "the cost is nothing" situation and the candidate refusing it for some random company in a random stage of the interview is absolutely reasonable.
While I don’t disagree with you I find it to be a slippery slope to some extent.
Would you screen out Linus Torvalds because he hypothetically doesn’t want to come in to a physical office for an interview?
Hiring managers should think long and hard in a data-driven way about whether the office presence is so necessary that you are willing to miss out on the best candidates who have the luxury of being picky.
Is it true scientifically that an in-person interview day results in better candidate quality or is that just a vibe?
I think eliminating top talent who refuse to step foot in an office and are rare enough to be able to maintain that demand is a lot of quality people being left out of your talent pool. I thought during the pandemic we already proved by numerous studies that in-office workers are less productive.
My company philosophy would be more like, put the burden of identifying quality talent on the employer rather than the employee. Put the candidate through the minimum effort required to screen them and identify standout talent. Then when you find that standout talent you roll out the red carpet and focus on convincing them to work at your company.
You can come up with outlier examples of course--though I'm not sure how relevant they are unless you're looking at hiring a "name" for some reason. But I'd still default to an in-person visit of some sort. I've never seen any data but then in-person was just assumed in most cases until a few years ago.
Read your last two sentences over again. That’s exactly my point: it’s all an old habit that isn’t based on outcomes.
I think it’s a human social instincts thing and not a quantitative thing. There might be a better way but we default to the social ritual because ape brain is most comfortable with it.
I don't know about that. Long ago I interviewed with someone that wanted some trivial C++ thing written on their laptop. I hadn't seen a Windows dev machine before and had no Internet access. I think I'd worked out the compiler was called visual studio and how to compile hello world by the time limit. Not sure that told either of us much.