Thread by @undersequoias

Well, this looks mighty awful for the future of algorithmic accountability.

Others know a lot more about disparate impact legal matters, so I'll comment on the applied tech ethics side of it. revealnews.org/article/can-al…

2/ When I work with tech corps, I'm often stating that the practical effect of algorithmic governance is the retention of context. Everything about machine learning technical systems motivates toward the stripping of context, and without context you can't have ethical reasoning.

3/ Governance artefacts, like impact reports or product requirement documents, function to retain a narrative about how the product was built, including ethically relevant information, such as what features were used to build the model, how bias was tested, etc.

4/ Legislation like Algorithmic Accountability Act requires such documents be produced, which in turn requires the organizations to develop the technical and organizational capacity to produce them. That is, essentially, the labor of "ethics": show your work, do due diligence.

5/ A purpose of something like the AAA, or softer regs like IEEE standards, is to provide safe harbor to organizations that do the work but still mess up. The demonstrable due diligence shelters them from liability—that is how you make a biz case for investing in ethics capacity.

6/ However! The rules proposed by HUD to define liability for algorithmic housing discrimination achieves precisely the opposite. These rules seek safe harbor for those who *don't* test for discriminatory outcomes and instead rely on vague third-party determinations. The text:

7/ This model, especially if other agencies take it up, would dissolve whatever inroads we have around these issues. It would reward bad actors and disadvantage, or even punish, those who invest in the capacity to determine the consequences of their algorithmic products/services.

8/ Our collective goal should be to create standards for due diligence throughout the development process, the procurement process, and the supply chain. The HUD rule will create an economic incentive to ignore those nascent norms.

9/ Notable problem 1: The use of the term "substitute" both here and in the preamble, where "proxy" is the correct term. No one knows what "substitute" means.

10/ Notable problem 2: ML services are increasingly third-party. This offloads liability to enterprise platforms, but most of the enterprise platforms have no access to their customer's data. HUD is building the cracks for plaintiffs to fall through here.

11/ Notable problem 3: Reliance on "industry standard" models is nonsensical. These services are highly customized and tailored, that is their entire economic premise. A model that is not discriminatory in Ann Arbor may well be in Detroit, and now we wouldn't need to check.

12/ Notable problem 4: HUD says nothing about algo transparency. How are plaintiffs to know which proprietary model the bank bought from a third party, let alone which 10k features were input into that model? A winnable case will be impossible.