Thread by @normative

50 years into the era of personal computing, the idea that all computers should come preloaded with spyware had only really been seriously entertained by authoritarian regimes like North Korea. Apparently now it’s going mainstream. cato.org/blog/apples-ip…

I’m curious how far they’ve thought out the legal end of this. A government (ours or an uglier one) approaches Apple with a court order saying “here’s a list of hash values we want you to add to the scan list you’re pushing out”. Can they refuse? Or even tell anyone?

This isn’t really a “slippery slope” — it’s a single heavily greased step. You need one order with a gag attached saying “you’re required to add this list of hashes” & your carefully crafted child protection system becomes an all-purpose population-scale search tool.

And a whole bunch of the arguments Apple deployed in the San Bernardino encryption case don’t obviously apply if they’ve already built the scan architecture & a government is just adding items to a preexisting list.

I should add, because a couple folks have noted this: IF this were implemented only as Apple intends and for the purposes it states, then yes, this would be functionally identical to the sort of scanning that’s routine on platforms & cloud storage services….

…and indeed, arguably significantly more privacy protective than most. But that strikes me as an incredibly naive way to evaluate technological architectures against a backdrop of compelled assistance by states with wildly varying commitment to civil liberties.

If everything operates precisely as Apple intends, scanning client-side is at worst an accounting detail & at best enables greater privacy. If everything does not operate precisely as Apple intends, moving scans client-side is a dangerous Rubicon to cross.

One in-the-weeds detail: Apple’s current design does have the final step of the “match” process happening server-side. That is: The phone checks images for a match but doesn’t “know” it found a match, which requires a server-side secret. apple.com/child-safety/p…

This is good design! And if your baseline is the quite common “file is uploaded unencrypted & scanned server-side” (rather than zero-knowledge) you could argue it’s a privacy improvement. But I doubt it’s worth crossing the Rubicon of coopting the device into the process.