In my view there are three takeaways from this, all of which are correct:
1. Already strained government services need to bear increasing input, affecting waiting times and employee workloads, which is bad.
2. LLMs are empowering citizens to appeal in a social security landscape which often require expert-level knowledge to navigate, which is good.
3.This phenomenon says more about the complexity of our social security systems than it does about LLMs, hopefully prompting our governments to review and start thinking about social security reform.
>2. LLMs are empowering citizens to appeal in a social security landscape which often require expert-level knowledge to navigate, which is good.
Including Voter Roll Purge requests? That’s in the paper as well.
Child protective services has a tip line.
You forgot:
2b. LLMs are empowering bad actors to exploit loopholes and collect benefits they're not entitled to.
The assumption from a lot of commenters seems to be that these are all legitimate claims and appeals, not a gold rush to get money from the government while the getting's good. In the end it will be the good faith actors that suffer - both those relying on these benefits to survive who will have to jump through more hoops to prove it and those funding the system with their taxes.
Having older parents and watching what my grandparents went thru long before LLMs I just don't worry about the poor old state that much.
The political zeitgeist is to always blame the "bad people", the "welfare queens" and the "undeserving" as the problem with the system. But never the complex system that's seemingly designed to benefit things like corporations extracting as much wealth from the system as they can. Or politicians ear marking the money for things they want to do, rather than the people that put the money in the system.
So, yea, your reasoning just really doesn't hit me in the heartstrings at all.
>Having older parents and watching what my grandparents went thru long before LLMs I just don't worry about the poor old state that much.
You should worry more about your parents whose benefits are jeopardized by stress to the benefits system.
Is there any evidence that this is actually a real worry?
Don’t audits usually find that vast majority of benefits are valid and correctly applied?
Medicaid and medicare make up 50% of the "abuse" (primarily overpayment), which again points towards companies milking the system, not individuals https://washingtonstand.com/article/gao-finds-15-major-feder...
I decided to read the report this is article is referring to. The report is estimating improper payments which is basically just missing paperwork and other types of payment errors. The report says explicitly that it is not measuring fraud rates.
So I dug a little deeper: roughly 94% of Medicare/Medicaid dollars and 89% of SNAP dollars passed audit reviews in 2025, but I can't find any national percentages for beneficiary fraud rates overall (which is sort of what we are talking about).
Provider fraud does appear to be the main driver of healthcare losses if we look at the enforcement evidence, but that does not mean every improper payment is abuse. It does seem the programs are audited quite extensively (unlike what some other comments in this thread are claiming).
Stress to the benefits system != fraud
Stressing the benefits system includes improper payments, fraud, and even the perception of fraud or anything else that decreases political support. It also includes the effective denial of service that this paper documents.
It should not be controversial to expect high integrity for a system that pays out money. “Its OK, its just a minimum of $186B in improper payments in 2025, not fraud” shows that it doesn’t have high integrity and that you don’t expect it to.
I think we might be using the term "stress" in different ways.
This paper makes a distinction between direct operational strain and harm created by governments' responses (like adding fees, procedural barriers, and other types of friction in the system). The second part is where my concern is, when bureaucracy is meant to suppress questionable claims it can also delay or deter legitimate benefits.
The paper says it doesn't/can't make definitive claims about AI's impact, but says the strain is moderate. The conclusion does not seem to suggest any severe operational risk to these agencies/systems. Only 9 of 84 potential cases involved benefits or social protection, and it says most cases are good-faith users.
So, I do question what in the paper would show that the burden is large enough to jeopardize benefits? The main finding seems to be that friction in the system may itself effectively deny legitimate access.
Response to your edit: the $186 billion is GAO's government-wide estimate across 64 different programs and not a fraud figure or addressing individual benefits. It contains administrative and document errors, indeterminate payments, and underpayments. And expecting integrity in a system doesn't prevent criticism in the form of asking whether added enforcement bureaucracies cause more harm to legitimate beneficiaries than it prevents in that system.
$186B are overpayments. That stresses the system. It's $186B you cant pay out legitimately and draws the ire of tax payers.
No one ever audits them.
That can’t possibly be true since I’ve read audit reports on public benefits
>But never the complex system that's seemingly designed to benefit things like corporations extracting as much wealth from the system as they can.
What “system”? Are you talking about? Social security? How is social security designed to maximize corporate wealth extraction?
If you mean the more nebulous “system” (“the system, man”), or capitalism, etc. then you’re equivocating.
is that actually happenng?
"No, but we're reporting it is" --southpark
> hopefully prompting our governments to review and start thinking about social security reform
I see the opposite, where LLMs and purchased datasets will make it easier for governments to create additional barriers for taxpayers seeking welfare.