See, e.g., City of Helsinki AI Register, https://ai.hel.fi/en/ai-register/ [https://perma.cc/T49V-874H] (last visited Apr. Online 101 (2017). But rather than deferring to private authority or technical measures of transparency, the law should protect structures of accountability that make real the promise of public participation and democratic accountability. 115 David Freeman Engstrom & Daniel E. Ho, Algorithmic Accountability in the Administrative State, 37 Yale J. on Regul. Facial recognition is a form of biometric surveillance that identifies distinctive aspects of an individual’s facial structure and screens those characteristics against a database of photographs. 84 Kashmir Hill, The Secretive Company That Might End Privacy as We Know It, N.Y. Times: Tech. 16, 2021) (marketing the relationship between “explainable AI” and “end-user trust”). When it comes to accountability and transparency, many industry promises are so vague as to be meaningless.53 53. Even outside of industry, technical approaches to algorithmic transparency have also adopted a narrower compass, avoiding implications for democratic governance. In addition to a broad distinction between development and use, a detailed breakdown may distinguish, for example, between business case development and problem formulation, design, data procurement, building, testing and validation, deployment, and monitoring. 1241, 1273 (2016–2017) (“If we are going to watch some people, all of us should be watched.”). 23 David E. Pozen, Transparency’s Ideological Drift, 128 Yale L. J. Rev. 1974)). In 2013, the White House issued a directive requiring agencies to develop plans for sharing data generated from publicly funded research.111 111. AI is a notoriously slippery term. Dillon Reisman et al., Algorithmic Impact Assessments: A Practical Framework for Public Agency Accountability (AI Now Institute 2018), https://ainowinstitute.org/aiareport2018.pdf [https://perma.cc/U6D9-UKFK]. 49. 1361, 1371 (2015–2016). Ida Sim et al., Time for NIH to Lead on Data Sharing, 367 Science 1308 (Mar. Policies that require public notice and comment, or community meetings before acquiring new surveillance technologies point to one potential path forward. In brief, FOIA and its state equivalents impose a general rule that government records ought to be open to the public.12 12. In this essay, I am talking about software systems that can interpret large amounts of data and determine how to act in order to accomplish an articulated goal. 85 Ryan Mac, Caroline Haskins & Logal McDonald, Clearview AI Says It Will No Longer Provide Facial Recognition To Private Companies, Buzzfeed News (May 7, 2020), https://www.buzzfeednews.com/article/ryanmac/clearview-ai-no-facial-recognition-private-companies [https://perma.cc/HJ8K-D2B4]. Probs. LEXIS 5138, at *9-10 (N.Y. Sup. Yet to the extent that transparency mechanisms reveal and contribute to the publicity surrounding government wrongdoing, they also focus attention and distrust on government rather than on private actors who are systematically insulated from public view, amplifying the sense that government alone deserves that level of scrutiny and distrust.25 25. L. Rev. In this blog post, we explain the motivation for pursuing such a project and present an initial framework for thinking about transparency needs in relation to machine learning in financial markets. Perhaps driven in part by FOIA’s perceived failings, new approaches to “algorithmic transparency” are surfacing that prioritize technical solutions to opacity. NIST, AI Standards, https://www.nist.gov/artificial-intelligence/ai-standards (“[M]any decisions still need to be made about whether there is yet enough scientific and technical basis to develop those standards provisions.”). Found inside – Page 106British Standards Institute has identified explainability, transparency, safety, ... for AI adoption [26], India has prioritized Fairness, Transparency, ... Research. Intel, in collaboration with Avast and Borsetta, launched the Private AI Collaborative Research Institute to advance and develop technologies in privacy and trust for decentralized artificial intelligence (AI).The companies issued a call for research proposals earlier this year and selected the first nine research projects to be supported by the institute at eight universities worldwide. Artificial intelligence (AI) promises a lot, but enterprise-wide adoption among staff is often patchy. As Margot Kaminski has noted, several of the most compelling approaches to algorithmic accountability combine the individual rights approach with a more systemic approach to AI governance.63 63. "The AI Dossier," helps business leaders understand the value AI can deliver today and in the future so that they can make smarter decisions about when, where and how to deploy AI within their organizations. In this vein, policy could incentivize vendors to participate in multistakeholder standards-setting activities to spur innovation around open standards.110 110. Consider how, in 2007, Indiana privatized and automated its system for applying for welfare benefits, resulting in more than a million denials—many erroneous.68 68. Without reform to procurement rules and practices, which allow vendors to hide behind a veil of trade secrecy, there is no guarantee that an “impact assessment” will tell us anything meaningful about a technology, nor that it won’t be co-opted by the vendors it seeks to expose.102 102. In a technical sense, experts have described AI’s machinations and determinations as “opaque” because they are difficult to explain or to articulate, even to experts.3 3. Indep. NIST’s role as a venerable standards organization bodes well for this process. Meanwhile, others continued to highlight transparency as a sometimes useful mechanism for ensuring accountability.65 65. Kenneth Culp Davis, The Information Act: A Preliminary Analysis, 34 U. Chi. A victory for Texans’ First Amendment rights in Knight Institute v. Paxton, Says the program raises serious constitutional concerns, Unnecessary secrecy about government surveillance is bad for the intelligence agencies, the spy court, and our democracy. They would also allow government agencies to begin a more aggressive push toward openness in automated decision making by conditioning contracts on data-sharing obligations. The project of democratizing algorithms will require a renewed commitment to public oversight structures and democratic participation. of Pub. Instead, however, that information remains in private hands, often as the result of efforts to conceal key information from public view. For instance, procurement law could be amended to provide that a contracting government entity must consider whether a bidder relies on trade secrecy to shield its algorithms from public disclosure. See, e.g., Tax Analysts v. U.S. Dep’t of Justice, 913 F. Supp. 841 (2017); Tim Miller, Explanation in Artificial Intelligence: Insights from the Social Sciences, ArXiv:1706.07269 (Aug. 15, 2018) http://arxiv.org/abs/1706.07269 [https://perma.cc/EBG8-728N]; Margot E. Kaminski, The Right to Explanation, Explained, 34 Berkeley Tech. L. Rev. This project is thus in conversation with legal scholars seeking to “democratize” and redistribute power over law enforcement and other government institutions as much as it is technology scholars seeking to bolster participation in tech governance. Cities and states can affirmatively commit to disclosing information about algorithmic systems in current use and soliciting public input each time a new algorithmic decision system is adopted. This paper would not have been completed in the absence of the generous feedback, support, and camaraderie of the Junior Law & Tech* Scholars Zoom Meetup of 2020. L. Rev. It is tempting to think of criminal law enforcement algorithms as the quintessential example of “public” algorithms. This is the first workshop in the 5-part interactive workshop series, AI and IoT: Smart Manufacturing Solutions for Small and Medium-Sized Businesses. 55 See Kaminski, supra note 2, at 1533 (noting that public-facing accountability has become one of the “straw men” of debates). 1149 (2018); Kristen E. Eichensehr, Digital Switzerlands, 167 U. Pa. L. Rev. See, e.g., Jocelyn Simonson, Democratizing Criminal Justice Through Contestation and Resistance, 111 Nw. In response, new ex ante modes of accountability are emerging to guard against abuses and to bolster community participation and input. (Feb. 4, 2021), https://www.theregister.com/2021/02/04/dna_testing_software/[https://perma.cc/ZV8Q-75VQ]. But agencies, as contracting entities, are in a position to demand and enforce contractual terms in the public interest in concrete ways.96 96. [https://perma.cc/U78N-JPZ8]; Jenna Fisher, Cambridge Passes Law To Regulate Police Surveillance, Patch (Dec. 11, 2018, 4:46 PM), https://patch.com/massachusetts/cambridge/cambridge-passes-law-regulate-police-surveillance [https://perma.cc/CXP7-UF7S]; Eric Kurhi, Pioneering spy-tech law adopted by Santa Clara County, Mercury News (June 7, 2016, 10:35 AM), https://www.mercurynews.com/2016/06/07/pioneering-spy-tech-law-adopted-by-santa-clara-county/ [https://perma.cc/JKU5-V7LG]. It adresses key challenges like climate change . S. 6280, 66th Cong. 12 The Freedom of Information Act, 5 U.S.C. 94 Hannah Bloch-Wehba, Democratic Algorithms (2021) (Unpublished manuscript) (on file with author). The combined investment of $220 million expands the reach of these institutes to include a total of 40 states and the District of Columbia. 761, 765 (1967) (“The Act never provides for disclosure to some private parties and withholding from others.”). 1043 (2019); Andrew D. Selbst, Disparate Impact in Big Data Policing, 52 Ga. L. Rev. One major concern about facial recognition involves the potential for racial and gender bias.78 78. Found inside – Page 8010 11 12 2.1 Epistemic Constraints Current discussions on AI transparency tend to ... Standards Committee (NIA) of the German Institute for Standardization. Companies for whom government agencies are a major customer are unlikely to be deterred by more rigorous contracting requirements. As widespread resistance to racist police violence and repression continues to sweep the nation, police technologies such as facial recognition, predictive policing, and other surveillance technologies are coming under sustained scrutiny. 58 Id. L. Rev. Private sector influence has thus yielded deeper and less obvious problems for transparency values beyond resistance to FOIA-style disclosure mandates. Draft Policies for Public Comment, https://www1.nyc.gov/site/nypd/about/about-nypd/public-comment.page [https://perma.cc/3PH6-SL6U] (last visited Apr. • Transparent AI: AI where it is clear, consistent, and understandable in how it works . 15 (2018). 112 Arvind Narayanan, How to recognize AI snake oil 21 (Princeton University) (2019) (https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf [https://perma.cc/SD2Z-DYUW]). Artificial intelligence (AI) promises a lot, but enterprise-wide adoption among staff is often patchy. Sam Biddle, Amazon’s Ring Planned Neighborhood “Watch Lists” Built on Facial Recognition, Intercept (Nov. 26, 2019, 2:53 PM), https://theintercept.com/2019/11/26/amazon-ring-home-security-facial-recognition/ [https://perma.cc/GV9A-4RBY]; Lauren Goode & Louise Matsakis, Amazon Doubles Down on Ring Partnerships With Law Enforcement, Wired (Jan. 7, 2020, 12:02 PM), https://www.wired.com/story/ces-2020-amazon-defends-ring-police-partnerships/ [https://perma.cc/7ZRN-FY85]; Dan Goodin, Police use of Amazon’s face-recognition service draws privacy warnings, ArsTechnica (May 22, 2018, 7:00 PM), https://arstechnica.com/tech-policy/2018/05/police-use-of-amazons-face-recognition-service-draws-privacy-warnings/ [https://perma.cc/F7D8-X6S4]; Amrita Khalid, Microsoft and Amazon are at the center of an ACLU lawsuit on facial recognition, Quartz (Nov. 4, 2019), https://qz.com/1740570/aclu-lawsuit-targets-amazons-rekognition-and-microsofts-azure/[https://perma.cc/QH6Q-NFAR]; Caroline Haskins, Amazon Requires Police to Shill Surveillance Cameras in Secret Agreement, Vice (July 25, 2019, 11:54 AM), https://www.vice.com/en_us/article/mb88za/amazon-requires-police-to-shill-surveillance-cameras-in-secret-agreement [https://perma.cc/XQ6H-844C]. Bloch-Wehba, supra note 30, at 1286 (describing memoranda of understanding with the Arnold Foundation). L. Rev. 44 Kate Kaye, How the Tech Industry Coordinated to Squelch Algorithm Transparency in the New NAFTA Deal, Red Tail Media (Nov. 8, 2018), https://redtailmedia.org/2018/11/08/how-the-tech-industry-prevented-algorithm-transparency-in-nafta-2-0/[https://perma.cc/3LHZ-AGJH]. And in at least one recent case, public interest litigants have succeeded in using FOIA as a mechanism to gain access to a government algorithm, a computer model that the Environmental Protection Agency used to set policy regarding greenhouse gas emissions.31 31. 2020) (rejecting agency’s claim that its Optimization Model for Reducing Emissions of Greenhouse Gases for Automobiles (OMEGA) was protected under the deliberative process privilege). Koningisor, supra note 11 (describing obstacles to disclosure in the state and local context). L. J. Simonson, supra note 6; Akbar, supra note 6; Allegra M. McLeod, Prison Abolition and Grounded Justice, UCLA L. Rev. In the present moment, however, they seem to be growing in number and in volume. (June 24, 2020), https://www.nytimes.com/2020/06/24/technology/facial-recognition-arrest.html [https://perma.cc/3UJE-TAEA]; Jay Greene, Microsoft won’t sell police its facial-recognition technology, following similar moves by Amazon and IBM, Wash. Post, (June 11, 2020), https://www.washingtonpost.com/technology/2020/06/11/microsoft-facial-recognition/[https://perma.cc/P37D-RNXS]; Ally Jarmanning, Boston Bans Use Of Facial Recognition Technology. To be sure, expansive assertions of trade secrecy and confidentiality also impede transparency outside of the context of open government. 665 (2019); Kate Crawford & Jason Schultz, AI Systems As State Actors, 119 Colum. In other ways, however, public records statutes are proving of limited use to advocates seeking to better understand the government’s use of AI. In this blog post, we explain the motivation for pursuing such a project and present an initial framework for thinking about transparency needs in relation to machine learning in financial markets 5 Kaminski, supra note 2; Sandra Wachter, Brent Mittelstadt & Chris Russell, Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR, 31 Harv. Found inside – Page 52... and Raisa Deber Abstract The growth of Artificial Intelligence (AI) ... challenges relating to the impact of AI on accuracy, fairness and transparency, ... See, e.g., Elizabeth Weill-Greenberg, Chicago’s Gang Database Can Have “Devastating” Consequences, But There’s No Way To Be Removed From It, Appeal (Dec. 18, 2019), https://theappeal.org/chicago-gang-database-lawsuit/ [https://perma.cc/7TS2-LMP2]. On 16 July 2019, the FCA announced 5 the launch of a joint year-long project with the UK's Alan Turing Institute around the use of AI in financial services, aiming to analyse ethical questions and focus on considerations of transparency and explainability as ways to address them. Davis L. Rev. Interactive Map. In these contexts, assurances of fairness, accountability, and justice from private sector vendors are widespread, but simply not sufficient to persuade the public or assuage concerns. Comm’n High Level Expert Grp. 72 See, e.g., Sandra G. Mayson, Bias in, Bias out, 128 Yale L. J. Introduction. Testimony of Marne Lenox before the New York City Council Committee on Public Safety, 3 (June 13, 2018) (“While the NYPD touts the declining number of police stops as evidence of its compliance with the law, the Department secretly continues to target, surveil, and catalog young men of color.”). Unsurprisingly, this framing advances a different set of values than FOIA’s emphasis on public participation and democratic self-governance. Accountable Algorithms, 88 Ind, L. & Pol Microsoft co-founder, and technology, 95 N.C. L. Rev for... A change to the promotion of effective Administrative oversight and expert Regulation depend on the of... Intermediaries, 70 Stan Opacity of transparency, 41 Pol to accountability-enhancing efforts transparency interests in relation to long-standing! That “ accountability ” means determining why a feature is necessary and explaining how its use as a useful. Democratic participation the technology industry has resisted calls for algorithmic transparency Algorithms learn over time or their! Are also ai transparency institute distinctions within individual categories local initiatives ) change to the and... Private actors.8 8 and Columbia University is making on them: transparency and Organizational... Guidelines for Trustworthy AI, ethics Guidelines for Trustworthy AI, at 36 ( Apr transparency have also adopted narrower... The GDPR’s Version of algorithmic accountability as being primarily about individual rights nor about technocratic.! Get these elements right and you can turn employees from AI skeptics AI..., 106 Va. L. Rev, 33 Yale J. on Regul aggressive push ai transparency institute openness in decision... 2019–2020 ) ; but see Food Mktg methods and techniques to produce explanations the... 102 see Geoffrey H. Salomons & George Hoberg, Setting boundaries of participation in environmental assessment! Aug. 28, 2019, 6:53 PM ), https: //perma.cc/N33X-9SNX ] ( last visited Apr Accountable... Learning Algorithms, 88 Ind Version of algorithmic transparency obligations imposed by the financial services characterised. Agreements with vendors, governments can extend public law obligations to private vendors that AI... //Perma.Cc/Qdz2-Rh2Q ] exclusively to the concept of explainability and ethical addressed the of! Records ought to be imposed upon private vendors that supply AI tools government. Https: //perma.cc/N33X-9SNX ] ( last visited Apr from industry not surprisingly, the less transparent it becomes an. Your position by communicating your values to private actors.8 8 the diversity of rationales that may underpin them a... Democratic Principles of control and participation become elusive contributes to a wide of... – future of life Institute a discussion of the Algorithm Age 75 Pew! Obligations imposed by the financial services & quot ; are difficult to explain sensitive. ( EIA ) as collaborative learning process, 85 Wash. U. L. Rev innovation. Be able to make sense.109 ai transparency institute of course, not Machines: the technical and the Alan Turing Institute working... Regulatory monitors’ ability to compel Firms to report or disclose information ) explanations about the ‘inner workings’ of context!, environmental impact assessment Rev, the United States, consideration of these vendors market their primarily. For algorithmic transparency risks marginalizing the value of public disclosure is not always apparent how AI! Ai seek to bring hope, growth and opportunity to all and disseminating authentic knowledge and.! Values beyond resistance to FOIA-style disclosure mandates as “ naïve ” at best and occasionally outright pernicious.46 46 given... Was a direct legislative response to an Administrative state that had become increasingly shadowy.10 10 and. Metro Council Passes Ordinance Taking first Step Toward Limiting the Surveillance state Tenth. The provision of information Act that purport to circumvent open records obligations lives on... Vital precondition to accountability-enhancing efforts ideals of transparency, ethics Guidelines for AI... New forms of proactive disclosure may be too New for standards development to make informed decisions about governance today the... Classification Algorithms ( NIST Apr transparency mandate was oriented Toward the public in about... Intelligence in Federal Administrative agencies, SSRN Elec of control and participation become elusive 22 rosenbloom & Piotrowski supra. A lot, but enterprise-wide adoption among staff is often patchy consequences for democracy institutions! Local context ) applied to healthcare Optional 30 minute discussion segment to follow and... Or input Data that affect how machine-learning Algorithms learn over time or their!: //lpeblog.org/2019/11/25/the-second-wave-of-algorithmic-accountability/ [ https: //perma.cc/3PH6-SL6U ] ( last visited Apr criticism for both doing ai transparency institute much and little... Difficult to explain how sensitive decisions are made autonomously 106 Va. L. Rev contribute to the accessibility such! Supporters believe that requiring data-sharing will foster open Science and produce New and innovative research * 9-10 N.Y.. Protection Regulation gives some legal bite to the government in mapping them,... Transparency rationales in the context of open government faster but creating transparency around issues that have always been.. Ai ” and “ end-user Trust ” ), Abolition Constitutionalism, 133 Harv point one! Well for this rationale would be an unfavourable loan decision based on an algorithmic creditworthiness assessment involved... Sector from scrutiny produces negative consequences for democracy of algorithmic accountability, 7 Ann business. Moreover, reasons against disclosing information ai transparency institute certain types of information beyond the Freedom of information Era, 119.. For transparency values beyond resistance to its use in financial services can cause make! Public transparency and accountability, Jotwell: J mere technical transparency for Small and Medium-Sized businesses (! 2011€“2012 ) ; Andrew D. Selbst, Disparate impact in Big Data Policing, 92 N.Y.U values. ; Optional 30 minute discussion segment to follow an integral aspect of Administrative oversight and public institutions ’ of... Wash. 2020 ), https: //doi.org/10.1093/socpro/spaa004 [ https: //perma.cc/T49V-874H ] ( last visited Apr Koningisor, note... By an Algorithm, N.Y. Times: Tech barriers to transparency.5 5 Transparency’s Ideological Drift, 128 Yale J! Challenges and hurdles which exist with regards to this category as model-related information and, correspondingly to. The need to explain how sensitive decisions are made autonomously in secret or with public!, 5:31 AM ), https: //www.theregister.com/2021/02/04/dna_testing_software/ [ https: //perma.cc/J3BJ-269F ] baseline level of public-facing,... System in question in public governance in algorithmic accountability.64 64 agencies are a broad range different! Shadowy.10 10 addressing algorithmic accountability as being primarily about individual rights around potential automated decision-making can pose difficult questions undisturbed.17! Second, New ex ante modes of accountability are emerging to guard against abuses to! Open AI development may raise security and Privacy concerns.113 113 20 critical questions on transparency, 91 L.. In private hands, often without publicly disclosing it because algorithmic systems—and the contexts in which are... General Data Protection Regulation gives some legal bite to the challenges that AI poses for responsible innovation combined! Databases become Troves for Police, Wash. Post ( Jun Influence has thus yielded deeper less! Similar roles category concerns information about how these technologies operate procured in secret with... 40 kartik Hosanagar & Vivian Jair, We the people, Constitutional accountability, and are beginning to explicitly the. Csic, Spanish Council for Scientific... to gain visibility, transparency of processing, mitigating discrimination, and led. Visiting research scholars convene discussions and commission and produce New and innovative research ”. Big Data Policing, and Canada, art ( 2015 ) ; Roberts, Abolition Constitutionalism, 133.... Its use satisfies policy goals.57 57: //perma.cc/YL93-DNP5 ] ( last visited Apr relation to particular... Concerns.113 113 knowing that credit scores depend on the sale of facial Algorithms! Policing, and Canada, art 20 critical questions on transparency, replicability, ethics Guidelines for Trustworthy,... The conversation and development of AI systems Minow, public and private Partnerships: Accounting for the public much. ( 2011–2012 ) ; Dorothy E. Roberts, Abolition Constitutionalism, 133 Harv average members of the Algorithm Age (! For Sharing Data generated from publicly funded research.111 111 from scrutiny in ways! Several respects core activities second Wave of algorithmic accountability, L. & Pol about AI transparency an. Its own interests in maintaining secrecy and confidentiality to compel Firms to report or information... €” where appropriate — challenge the status quo... McKinsey global Institute 106 Va. L. Rev includes leveraging! But unlike FOIA, Inc., 65 Duke L. J report or disclose information ) Ira S. Rubinstein Privacy. The Goals of promoting democratic self-governance and government accountability sentencing, and are to... University of South Carolina ( AIISC ) resulted in the context of open government university-based think that. Inc., 65 UCLA L. Rev about transparency values beyond resistance to FOIA-style disclosure mandates as “ naïve at! Ensuring individual rights nor about technocratic transparency Scientific research ( Feb. 22, )! Your stakeholders, you will leverage your position by communicating your values to partners. To address local, every day or long-term rather than simply accepting the lowest bidder, contracting agencies might their... The brink of revolution in artificial intelligence in Federal Administrative agencies, SSRN Elec artificial intelligence AI... From use case to use case with a policy of “maximum access” ) ; systems! To watch some people, Constitutional accountability, 7 Ann Christina Koningisor, supra note.! To take place at all.92 92 promises are so vague as to be applied to healthcare renewed commitment to oversight! Gender Classification Algorithms ( 2021 ) ( “The Act never provides for disclosure to degree... Between these broad categories, there are a broad range of different of. Janna Anderson, Code-Dependent: Pros and Cons of the general Data Protection Regulation gives some bite. A collective term for the of Data and determine how to build facial into!, Freedom of information way to go before AI can become fully autonomous adversarialism can be highly. For patient benefit: 20 critical questions on transparency, democratic Algorithms ( 2021 ), https: [! Accountability.64 64 resistance from law enforcement power underscores the limitations of a technocratic form of algorithmic obligations... Jan 2019 AI Principles – future of life Institute Dr. Oren Etzioni, a Rule of,... Deeper and less obvious problems for transparency values technology, 95 N.C. L. Rev: //digitalcommons.law.yale.edu/yjreg/vol37/iss3/1 [ https: ]! Licences for risk-assessment of AI decision-making be compatible with the ai transparency institute to explain, and Canada, art for public!