I actually just built an ATS for my company Mathpix. But it never occurred to me to use resumes. Basically we have a set of company values and a specific open ended questionnaire to gauge the fit:
https://mathpix.com/careers/apply
Then internally we have dashboards and sorting based on AI agent scoring. I noticed the scoring is imperfect but still saves a lot of time. Candidates scored at or below 2/5 are reliably bad and candidates above 4/5 are consistently impressive and leave thoughtful answers.
The biggest thing is not using resumes. You can’t reliably gage applicants without a writing sample and resumes are the worst form of writing sample. Also you need to be intentional about who you’re hiring for, both to craft the questions as well as grade the responses.
This seems likely to be worse. How do you screen out people who point an LLM at your values and ask it to answer your questions in a way likely to appeal to a recruiter using an LLM to score the responses?