US lawmaking has a distinctive failure mode: because of the Constitution's absolute language and extensive jurisprudence, lawmakers can please their base by enacting bad, overreaching or stupid laws and then hope the courts will narrow or overturn them before they detonate.
1/
1/
This moral hazard is not evenly distributed: if you are the party that decries "activist judges" and campaigns on the idea that governments are bad at everything, then enacting bad laws and then having them overturned serves your cause especially well.
2/
2/
On a totally different subject, let's talk about Ronald Reagan. After Reagan saw Matthew Broderick's classic technothriller WAR GAMES, he became convinced that America needed a far-reaching cybercrime bill, something Fed prosecutors had been demanding for years.
3/
3/
That's how the Computer Fraud and Abuse Act (#CFAA) came into being: it's a maddeningly badly drafted, overreaching, vague law that potentially felonizes any act that "exceeds your authorization" on someone else's computer system.
4/
4/
Private firms have taken the extreme position that since their terms of service define your "authorization" on their computers, that any violation of the terms of service is a jailable felony.
5/
5/
In practical terms, that means that if you violate a company's terms of service - a sprawling garbage-novella of deliberately impenetrable legalese - they can send you to prison, for a very long time. This is really bad.
6/
6/
Most of the time, of course, Fed prosecutors don't like to charge people criminally for violating ToS, but when they have someone they want to punish for petty reasons they can find a ToS violation and charge them criminally.
7/
7/
That's the @aaronsw story: Aaron violated MIT and JSTOR's terms of service, and a prosecutor that Aaron had previously humiliated by beating a bullshit charge was able to re-charge Aaron with 13 felonies and threaten him with 35 years in prison.
8/
8/
(Background: Aaron published a trove of paywalled, public domain court records from PACER, the feds' legal repository. He embarrassed the legal system by showing that these court records that anyone could get at $0.10/page were improperly redacted and exposed crime victims)
9/
9/
(Aaron later scraped a bunch of scientific journal articles he was allowed to access via MIT's network; but the system's ToS said he had to access them manually, not via a small script that downloaded them automatically - this was the felony)
10/
10/
(After using legal maneuvers to draw out the case until Aaron and everyone he could tap was broke, the PACER prosecutors were steaming towards a prison sentence for Aaron; he hanged himself rather than face incarceration)
11/
11/
Over the years the CFAA has had many court cases, and these have produced a "circuit split," with some US courts interpreting CFAA narrowly, and others taking a dangerous, expansive view of its text.
12/
12/
Ever heard the phrase "hard cases make bad law?" The thing about overreaching, vague laws like CFAA is that they can be shaped to criminalize ANY conduct, so if there's someone who did something objectively terrible, vague laws give prosecutors an easy path to "justice."
13/
13/
Nathan Van Buren is an accused dirty cop who sold access to license plate databases to his confederates. Prosecutors decided to charge him under the CFAA, which could indeed mete out severe punishments for this kind of bad behavior.
eff.org/cases/van-bure…
14/
eff.org/cases/van-bure…
14/
But that punishment comes at a high price: a precedent that could be wielded against ANYONE who violated Terms of Service, something that all of us do, a hundred times a day, without noticing it. It would give prosecutors leeway to do what they did to Aaron, over and over.
15/
15/
In support a briefs, a group of legal, security, AI, and human rights scholars published "Legal Risks of Adversarial Machine Learning Research," exploring the potential impact of Van Buren on the critical work of analyzing machine learning models.
arxiv.org/pdf/2006.16179…
17/
arxiv.org/pdf/2006.16179…
17/
Adversarial Machine Learning is the vital process of systematically testing machine learning models to reveal security defects, bias, and other problems. It is high-stakes work: without AML, you can trick car autopilots into steering into oncoming traffic!
18/
18/
AML is also key to revealing racial bias in risk analysis software, facial recognition, predictive policing, hiring algorithms, and a host of other areas in which peoples' freedom, prosperity, safety and very lives are at stake.
19/
19/
The authors explain in admirably plain language how a bad decision in Van Buren puts this enterprise at risk - how it could leave us (literally) flying blind, forced to rely on self-serving assurances of vendors when we trust their systems with every aspect of our world.
20/
20/
This is the worst possible outcome of the moral hazard in American lawmaking: not merely that lawmakers will promulgate bad laws to feed their base in the hopes that courts will strike them down and give them fresh grievances to campaign on.
21/
21/
But rather that these laws will become institutionalized, that they will give rise to questions so technical and nuanced that they slide through the courts and end up enshrined in our justice system.
22/
22/
Depending on the outcome of Van Buren, the CFAA could become an enduring tool for thin-skinned corporate execs and petty, vengeful prosecutors to imprison anyone that displeases them - including the security researchers we rely on to vet our increasingly automated world.
eof/
eof/
