I think the main takeaway is that there hasn't been enough push back and that this now seems depressingly inevitable.
I expect we will see more calls for surveillance like this in the coming months heavily remixed into the ongoing "online harms" narrative.
Without a strong stance from other tech companies, in particular device manufacturers and OS developers, we will look back on the last few weeks as the beginning of the end of generally available consumer devices that don't conduct constant algorithmic surveillance.
Which is quite a statement for me to make. I've spent the last few years heavily focused on metadata surveillance as I've seen that as the primary privacy threat given the rise of end-to-end encrypted messaging.
With on-device surveillance, the game has changed.
I abhor privacy nihilism but if you are willing to let a robot sit on your phone (or in your house) and report back on your activities then you have consented to not having any privacy.
You are free to make that choice (at least for now).
• • •
Missing some Tweet in this thread? You can try to
force a refresh
The attack improvements come from considering temporal relationships (the probability of receiving messages over a given threshold in a period of time) instead of just over the lifetime of the system.
This can be devastating if false positive rates are poorly selected.
Someone asked me on a reddit thread the other day what value t would have to be if NeuralHash had a similar false acceptance rate to other perceptual hashes and I ball parked it at between 20-60...so yeah.
Some quick calculations with the new numbers:
3-4 photos/day: 1 match every 286 days.
50 photos/day: 1 match every 20 days.
As an appendix/follow up to my previous article (a probabilistic analysis of the high level operation of a system like the one that Apple has proposed) here are some thoughts / notes / analysis of the actual protocol.
Honestly I think the weirdest thing given the intent of this system is how susceptible this protocol seems to be to malicious clients who can easily make the server do extra work, and can probably also just legitimately DoS the human-check with enough contrived matches.
Daily Affirmation: End to end encryption provides some safety, but it doesn't go far enough.
For decades our tools have failed to combat bulk metadata surveillance, it's time to push forward and support radical privacy initiatives.
Watching actual cryptographers debate about whether or not we should be voluntarily *weakening* encryption instead of radically strengthening threat models makes my skin crawl.
I don't think I can say this enough right? Some of you are under the weird impressions that systems are "too secure for the general public to be allowed access to" and it just constantly blows my fucking mind.
Based on some discussions yesterday, I wrote up a more detailed note on the Apple on-device scanning saga with a focus on the "obfuscation" of the exact number of matches and dived into how one might (probabilistically) break it.
This isn't the biggest problem with the proposed system. It does however suggest that even if you *really* trust Apple to not abuse their power (or be abused by power) then Apple still needs to release details about system parameters and assumptions.
We can quibble about the exact numbers I used, and the likelihood of the existence of a "prolific parent account" taking 50 photos a day for an entire year but there are *real* bounds on the kinds of users any static threshold/synthetic parameters can sustain.