By Mitchell W. Pearlman
Governments now gather almost incomprehensible amounts of information, organize it into vast databases, and make and implement important decisions using algorithms they create or purchase. An algorithm is the step-by-step procedure by which a task is performed. Algorithms used in computers are often highly complex and sophisticated. Because of this, algorithms are considered intellectual property and deemed trade secrets by most private enterprises and many government agencies that create and use them.
Governments use computer algorithms in making tax policy and budget decisions; they use them in forecasting various transportation and infrastructure needs; and they use them in analyzing public health, justice and environmental issues and in formulating policy based on these analyses. Of course, if the data used are less than complete or accurate, or if the algorithms themselves are based on flawed reasoning or assumptions, then government policies and decisions based on them will likewise be flawed. Such errors can lead not only to unsound decisions, but they also can lead to an enormous waste of public resources and even to a significant loss of life.
In Connecticut, various state agencies forecast income and expenditures to help guide lawmakers in constructing state budgets. Each of these offices has access to the same data sets. But the assumptions programmed into their algorithms can differ significantly, leading to different outcomes in determining whether a budget will or will not be in balance. For example, the 2018 state budget was out-of-balance by several hundred million dollars just weeks after it was enacted. How did this happen? Was it because the data was faulty? Or was it because the algorithms, and the assumptions built into them, were wrong?
To prevent such errors in the future, government algorithms need to be transparent so they can be publicly vetted before policy decisions are made or legislation becomes law.
The first shots in the battle for algorithmic transparency have already been fired. The New York City Council passed an algorithmic accountability bill, which establishes a task force to study how city agencies use algorithms to make decisions. The bill was enacted in the wake of a racially biased algorithm used to assess risk factors of criminal defendants. The algorithm’s source code was confidential until a federal judge ordered it to be disclosed and the bias subsequently identified.
Allegheny County Pennsylvania apparently has learned that government can no longer afford to treat algorithms as both secret and the exclusive domain of those who create and use them. According to Dan Hurley in a New York Times Magazine article (“Can an Algorithm Tell When Children Are in Danger?”), the Allegheny County child welfare agency stopped using an expensive algorithm developed by private companies. The algorithm was used to help screen cases of possible child abuse and determine which cases should be investigated within the agency’s limited resources. The algorithm did not perform well and its owners refused to reveal details to help the agency discover the problem.
The child welfare agency then replaced the faulty algorithm with a new one, which it developed with its own independent consultants and which the agency now owns. The algorithm was made available for all to see. Stakeholders – including government officials, technical experts, lawyers, child advocates, parents and even former foster children – were invited to discuss and comment on the algorithm before its adoption. Although not perfect, the new algorithm has been performing much better than the one it replaced, and most stakeholders speak highly of it.
Trade secrets and confidential commercial information often represent a significant financial investment by those enterprises and organizations that create or own them. On the other hand, computer algorithms are now – and increasingly will be – vital components in government policy and other decision making. To prevent significant errors or miscalculations in the future, many government algorithms need to be transparent so they can be publicly vetted before policy decisions are made or legislation becomes law.
In the case of algorithms used by government, proprietary rights face an important competing value when they would prevent the disclosure of information about which there is a legitimate and important public interest. The notion of an informed and knowledgeable electorate is one of the cornerstones of our nation’s democratic tradition. To paraphrase the Connecticut Supreme Court in another context, trade secrets and confidential commercial information must give way when balanced against the publication of matters of public interest, in order to ensure the “uninhibited, robust and wide-open discussion of legitimate public issues.”
In this instance, the balance of competing interests must be resolved in favor of algorithmic transparency to the greatest extent possible. This is not to say that government need not provide some measure of just compensation to private businesses if government discloses a business’ proprietary information. But the bottom line is that algorithmic transparency is essential to the continuance of our democratic system of governance.
[1] Edited from the CFOG White Paper “GOVERNMENT ALGORITHMS AND THE PUBLIC’S RIGHT TO KNOW,” March 11, 2018.