As I have said before, I am willing to be the person who draws a line here, against the calls for "nuance".
There is no room for nuance, because nuance thinks surveillance systems can be built such that they can only used only for good or to only target bad people.
It is our duty to oppose all such system *before* they become entrenched!
Not to work out how to entrench them with the least possible public outrage at their very existence by shielding their true nature with a sprinkling of mathematics.
I don't need to do the whole "what if" dance because I have seen it, and live it.
Surveillance is already here, but unevenly distributed.
I co-founded @OpenPriv so that I could work against it, every day.
There is a never ending stream of intentional surveillance flowing towards your right to privacy. It isn't just Apple. It isn't just corporations.
There are many large groups fighting tooth and nail to ensure you have less privacy today than you did yesterday.
The work that we do @OpenPriv is only made possible by the many small donations we receive each year from individuals like you.
I'm happy if you donate to any organization building privacy tech, but please consider checking out our work.
Here is the thing...since it has come up...we do "focus on real world harms". It is all we do. It is why we are so firmly, absolutely against the concept of surveillance.
There are plenty of terrors in this world, and most of them are forced on us by people with good intentions.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
Also has anyone else attempted to reverse engineer how Apple might have arrived at 1/trillion probability of false account flagging?
Some back of the napkin math, please double check...
If you assume the threshold is >10 false positives over a year to trigger an account (thrown around in the Apple docs), and each person stores ~1024 new photos per year (~3-4/day) then to get a 1/trillion figure your single-instance false positive probability has to be ~1/2000
You can get that probability if you assume the database being checked against contains ~16M unique hashes and the effective hash size is ~36bit (Neuralhash hashes appear to be 128 bit, but they are perceptual not random)
Neither of those values seems absurd given what we know.
These are fair question regarding systems like the one Apple has proposed, and there is enough general ignorance regarding some of the building blocks that I think it is worth attempting to answer.
But it's going to take way more than a few tweets, so settle in...
First, I'll be incredibly fair to Apple and assume that the system has no bugs - that is there is no way for a malicious actor inside of outside of Apple to exploit the system in ways that it wasn't meant to be exploited.
Idealized constructions only.
At the highest level there is your phone and Apple's servers. Apple has a collection of hashes, and your phone has...well tbh if you are like a large number of people in the world it probably has links to your entire digital life.
Really is disappointing how many high profile cryptographers actually seem to believe that "privacy preserving" surveillance is not only possible (it's not) - but also somehow "not surveillance" (it is).
Meanwhile Apple are making statements to the press with the effective of "We are not scanning peoples photos for illegal material, we are hashing peoples photos and *using cryptography* to compare them to illegal material"
As if those aren't the *EXACT SAME THING*.
It's very important to focus on the principles involved here and not the mechanism. Just because you use cryptography to alter the thing you are surveillance doesn't make it not-surveillance.
Clearly a rubicon moment for privacy and end-to-end encryption.
I worry if Apple faces anything other than existential annihilation for proposing continual surveillance of private messages then it won't be long before other providers feel the pressure to do the same.
You can wrap that surveillance in any number of layers of cryptography to try and make it palatable, the end result is the same.
Everyone on that platform is treated as a potential criminal, subject to continual algorithmic surveillance without warrant or cause.
If Apple are successful in introducing this, how long do you think it will be before the same is expected of other providers? Before walled-garden prohibit apps that don't do it? Before it is enshrined in law?
"Stop using encryption so we can can check your messages for criminal activity" becomes "Allow us to scan all the files on your computer for criminal activity"
I'm so tired. Maybe let's not do the dystopia of corporations building cop bots into general purpose computers.
At some point protecting your privacy is going to boil down to not using devices that actively spy on you. No amount of overlay software can protect you from making a bad choice there.
Don't support corps that scan your computer for crimes seems pretty fucking basic.
Today I spent 5 hours debugging, and finally moved a single line of code up 10 lines.
Reduced cpu usage 20x.
Just want that on the record.
Basically: something that was supposed to get called once per app run, was getting called on every top-level UI rebuild, and because it updated one of the main global settings providers that update was also triggering a top level UI rebuild - so there was a rebuild cascade.
Moved the call out to where it actually belongs...no more rebuild cascade, all the consumers are now caching properly, all the render boundaries are kicking in and everything is now ridiculous fast.