Using AI to create a first draft of your wedding program or a business plan for your Etsy shop might be the jump start you need to pull your ideas together into one place.
Using it to provide public services is quite another thing altogether. And sometimes not a good thing.
The Trump administration is encouraging federal agencies to use AI in its operations—for services such as tax audits, airport screening and air traffic control, caring for veterans, and even waging war.
At the same time, the administration is pushing many once-federal functions and programs back to the states, which will also likely begin to rely on AI because of the short runway for getting programs launched.
Cities and states are already incorporating AI into what they do—work that typically had been done by public sector workers, i.e., humans. We’re seeing governments use AI for important functions, such as risk scoring, benefits eligibility screening, fraud detection, and predictive policing. It doesn’t always go well.
GUNSHOT DETECTION
A number of cities have contracts with a company called SoundThinking to use the company’s ShotSpotter, a gunshot detection technology that relies on artificial intelligence and algorithmic methods to detect and locate gunshots. While SoundThinking purports to provide police a “consistent, rapid, and precise response” to gun violence, the city of Chicago found that the technology has been neither consistent nor precise. A study from the MacArthur Justice Center at Northwestern Pritzker School of Law examined 19 months (from 2019-2021) of ShotSpotter in Chicago and found that 89 percent of ShotSpotter’s reports led police to find no gun-related crime, and 86 percent of reports were not related to any crime. This resulted in 40,000 unfounded police deployments, which disproportionately impacted communities of color.
ELIGIBILITY SERVICES
Some states rely on contractors to administer social service eligibility services, and in some cases, these contractors rely on artificial intelligence to help them make decisions about a person’s eligibility for programs, such as Medicaid. Technical problems with these privatized automated systems have a history in numerous states of making incorrect decisions about an applicant’s eligibility status, denying low-income people and people with disabilities access to important health care. In Texas, 1.8 million people who were qualified for health insurance coverage, many of whom were children, lost Medicaid coverage in 2023 when states had to engage in the “unwinding” or removal of Medicaid pandemic protections. In 2024, healthcare and technology advocates filed an FTC complaint against Deloitte, a major contractor that builds and operates these systems, alleging that problems and glitches in their system erroneously caused many of these people to incorrectly lose their Medicaid coverage..
DROP-OUT DETECTION
Some school districts are turning to AI to help raise graduation rates. But these drop-out early detection tools, which utilize artificial intelligence through machine learning algorithms, are not always helpful and can actually hinder student progress. Some Wisconsin school districts have utilized a risk assessment system that gives students, starting in 6th grade, a score and risk level, based on the system’s assessment of their risk of dropping out of school before graduation. A study from The Markup found the assessment systems is wrong nearly 75% of time when it predicts a student won’t graduate on time, and it’s wrong at significantly greater rates for Black and Hispanic students than it is for White students. These types of risk labels can discourage and stigmatize students, and negatively influence how teachers perceive students labeled as high risk for dropping out. A study from UC Berkeley found that the Wisconsin system had almost no positive effect on graduation rates for students it labels high risk, undermining the argument for the adoption of the risk assessment system.
These are just a few examples, and there will likely be more as the use of AI spreads through governments at all levels.
Local Progress Impact Lab and AI Now Institute have produced a helpful guide to gaining a better understanding of the issues around public services and AI. Though aimed at local government officials, Local Leadership in the Era of Artificial Intelligence and the Tech Oligarchy offers a valuable primer for anyone concerned about AI. AI Now also published A Roadmap for Action, with ideas about combatting the consolidation of power that artificial intelligence could, if unchecked, lead to.
These are among the sources ITPI will be using as we keep an eye on AI and its uses—and misuses–that have an outsized impact on the public.
Shahrzad Habibi
Research and Policy Director