For Immediate Release l January 18, 2022
For More Information, Please Contact:
Stephen Stich 520-488-0486, (Yale) Stephen.stich@ylsclinics.org
William S. Fish, Jr. 860-331-2700, (CFOG) wfish@hinckleyallen.com
Michael Savino, 860-324-5768, (CCFOI) michael.a.savino@gmail.com
Three major CT state agencies have used automated software programs, called algorithms, to make policy decisions affecting school funding, removing children from families, and hiring state workers, but are unable or unwilling to fully disclose how these programs make their decisions.
Significant gaps in public oversight of “behind the scenes” computer decision-making were revealed in a comprehensive research report released by the Media Freedom and Information Access Clinic (MFIA) at Yale Law School, in collaboration with The CT Foundation for Open Government (CFOG) and the CT Council on Freedom of Information (CCFOI).
The report, Algorithmic Accountability: The Need for a New Approach to Transparency and Accountability When Government Functions Are Performed by Algorithms, is based on Freedom of Information Action (FOIA) requests made by the MFIA Clinic. The three state agencies — whose responses were generally deficient and untimely — are the departments of Children and Families (DCF), Education (DOE) and Administrative Services (DAS).
The report documents examples outside Connecticut where algorithms produced faulty background checks, made incorrect facial recognition matches, denied benefits erroneously, and wrongfully terminated parental rights. It sought to determine the extent to which the public can know that algorithms used by Connecticut agencies do not suffer from similar flaws. “The potential for real harm makes transparency surrounding the government’s use of algorithms critical,” said MFIA Fellow Stephen Stich, a co-author of the report. “This transparency does not exist in Connecticut today,” he added.
Speaking to the agencies’ noncompliance with FOIA, Mitchell W. Pearlman, former executive director of CT Freedom of Information Commission and CFOG officer, added, “Unfortunately, government agencies have always used these same techniques of delay, denial and obfuscation to hinder access to public records that would hold those agencies accountable and shed some needed sunlight on their activities.”
Algorithms are sets of computer instructions government agencies and others increasingly use to solve problems or accomplish tasks formerly done by humans. Policymakers use algorithms to conduct government business that has life-changing effects. They are used widely for everything from setting bail, to allocating police resources, and distributing social welfare benefits.
Although algorithms can improve effectiveness and efficiency, their use has problems, the report explains. Algorithms can make mistakes and worsen pre-existing biases. And when government algorithms make mistakes, they can upend lives.
The agencies’ foot-dragging and sparse productions led the Yale Clinic to conclude that “existing disclosure requirements are insufficient to allow meaningful public oversight of the use of algorithms, and that agencies do not adequately assess [algorithms’] effectiveness and reliability.” Moreover, agency personnel appear unconcerned about the lack of transparency, the report states.
Using the FOIA, the Yale students sought information about what certain algorithms used by the three agencies do; how they were obtained; whether they were evaluated; and the extent to which answers to those questions can be accessed under existing open-government laws.
Among the three state agencies, the most incomplete response came from DAS, arguably one of the most powerful state agencies, having broad authority over services that cover employment practices, procurement, facilities management, and technology.
Although FOIA requires state agencies to respond to inquiries within four business days, for several months DAS ignored the request for information about a new algorithm used in hiring state employees and contractors. Many attempts to set up meetings went unheeded. Ultimately, DAS provided no documentation after invoking dubious and irrelevant FOIA exemptions.
DOE responded, in part, to an inquiry involving an algorithm used to assign students to public schools, an issue hugely important given the court cases and settlements designed to end racial segregation in Connecticut schools. The agency failed to disclose how its school-assignment algorithm worked. There appeared no mechanism to allow parents to challenge its decisions.
Citing a trade-secret exemption, DOE refused to produce key data and failed to produce records regarding its acquisition of the algorithm except the procurement announcement and the contract (which totaled $650,000). Thus, it was impossible to evaluate the algorithm’s efficacy or bias.
DCF officials provided the only complete response to the Yale Clinic’s request, producing documents on its algorithm used to reduce the number of children experiencing life-threatening episodes. Illinois abandoned that same algorithm after it determined that it was ineffective. The report documented DCF had not performed a “robust evaluation of the algorithm’s efficacy or bias” before it was implemented or in the three years it was used.
The authors say legislative remedies are needed, like those introduced or considered by governmental entities elsewhere. The report discusses options that include forcing state agencies to assess publicly the effectiveness and bias of any algorithm used, mandating the waiver of trade secret protections in certain situations, and requiring disclosures to individuals who are subject to algorithmic decisions.
“While the details of these approaches require study, it is imperative that steps are undertaken now to identify an effective response to the current lack of algorithmic accountability. The potential for serious harm to be inflicted by malfunctioning or biased algorithms are too serious to ignore,” said William J. Fish Jr., president of CT Foundation for Open Government.
The report and executive summary are available at https://law.yale.edu/mfia/projects/government-accountability/algorithmic-accountability, and https://ctfog.org.