> So if a demographic group is more likely to apply to some jobs they are not qualified for, this paper would say they are being discriminated against.
Your understanding appears to be incorrect.
> Our research also found that this pattern does not appear to be the case in other circumstances. We analyzed data from the largest prior study of hiring decisions, which sent 83,000 applications to 108 Fortune 500 firms during the same time period as our study and did not focus on whether AI was used to make decisions. We found that the rate at which applicants were rejected from every firm they applied to in this data was no higher than what you’d expect if each company decided independently of the others.
If it were what you were asserting, then this behavior and results would persist even without AI being used. Instead when they remove the filter for AI decisions (and AI mono-culture in decisions) the effect is no longer present.
This seems to strongly support they argument that effectively a single AI makes a single decision for a candidate across "all" positions they apply for rather than independently assessing them for each position.
Essentially it's more or less saying they're is one hiring manager for the entire industry and if they have a random reason they don't like you, you won't be hired for any job in the industry.
There is a single evaluation function for the industry and if it puts you a negative for any reason in the model's distribution, every job that uses it will do so.
Those are somewhat separate concerns. You could have companies making independent hiring decisions while systematically discriminating against demographic groups, and you could also have companies all use the same system that systematically disadvantages certain individuals, but it's unrelated to their demographics and instead based on things like their resume not being easy to OCR.
In this case, the claim is that both are happening: companies aren't making decisions independently and they're doing so in a way that discriminates against certain demographics. But the evidence needed for each half of the claim is different.
> If it were what you were asserting, then this behavior and results would persist even without AI being used. Instead when they remove the filter for AI decisions (and AI mono-culture in decisions) the effect is no longer present.
> Our research also found that this pattern does not appear to be the case in other circumstances. We analyzed data from the largest prior study of hiring decisions, which sent 83,000 applications to 108 Fortune 500 firms during the same time period as our study and did not focus on whether AI was used to make decisions. We found that the rate at which applicants were rejected from every firm they applied to in this data was no higher than what you’d expect if each company decided independently of the others.
> If it were what you were asserting, then this behavior and results would persist even without AI being used. Instead when they remove the filter for AI decisions (and AI mono-culture in decisions) the effect is no longer present.
Thanks for this note, I missed this when skimming it. I would love to see their actual analysis here explained more than a single line, but this doesn't say the original study found no adverse impact at the job type level (they seem to say this wasn't analyzed), but rather that firms seemed to look more independent. Which makes sense for the headline, but is not about their notes on harms, which I still think have all the weaknesses I outlined.
> There is a single evaluation function for the industry
Could this be an opportunity in disguise? Somehow learn what this function wants, maximize it, then the entire industry opens up?
Apparently the key 'hack' is to present white or hispanic.
No idea why reiterating the exact point of the article got downvoted.