TW: Apple, CSAM
Plenty of incredible privacy researchers, cryptographers, sex workers, and others have talked about why Apple’s new CSAM detection tool is terrible in numerous ways.
I also want to quickly point out the abusability of this tool against adult survivors of CSAM.
Plenty of incredible privacy researchers, cryptographers, sex workers, and others have talked about why Apple’s new CSAM detection tool is terrible in numerous ways.
I also want to quickly point out the abusability of this tool against adult survivors of CSAM.
TW: Apple, CSAM
Many victims of CSAM will grow up and become adults. And here’s the thing: adults can make decisions including taking photos of their bodies whether sexually, for medical reasons, it’s not your business!
Do you see what I’m seeing here?
techcrunch.com/2021/08/05/app…
Many victims of CSAM will grow up and become adults. And here’s the thing: adults can make decisions including taking photos of their bodies whether sexually, for medical reasons, it’s not your business!
Do you see what I’m seeing here?
techcrunch.com/2021/08/05/app…
TW: Apple, CSAM
The lack of abusability testing of this tool and trying to be smart matching up to the existing NCMEC CSAM corpus means for many survivors you may find yourself losing autonomy again because an algorithm picked up on a unique identifier: birth marks, moles, etc.
The lack of abusability testing of this tool and trying to be smart matching up to the existing NCMEC CSAM corpus means for many survivors you may find yourself losing autonomy again because an algorithm picked up on a unique identifier: birth marks, moles, etc.
TW: Apple, CSAM
No one should expect survivors of CSAM to remember or know how CSAM of them were created to avoid collision with the tool because the algorithm is looking for similar photos.
No one should have to worry about hiding parts of their body if they don’t want to.
No one should expect survivors of CSAM to remember or know how CSAM of them were created to avoid collision with the tool because the algorithm is looking for similar photos.
No one should have to worry about hiding parts of their body if they don’t want to.
TW: Apple, CSAM
Got some follow up questions. To clarify, this is horrifying on a privacy level for everyone (I'll link to multiple well written threads on this later).
For survivors though the potential for collision with the algorithm is high since we don't know how it works.
Got some follow up questions. To clarify, this is horrifying on a privacy level for everyone (I'll link to multiple well written threads on this later).
For survivors though the potential for collision with the algorithm is high since we don't know how it works.
TW: Apple, CSAM
Traditionally, you use a hash of an image. This stays consistent if the same image gets distributed. Of course this hash can be changed in a myriad of ways. Apple's CSAM ML/AI algorithm allegedly can match the same visual image ignoring crops/distortions/etc.
Traditionally, you use a hash of an image. This stays consistent if the same image gets distributed. Of course this hash can be changed in a myriad of ways. Apple's CSAM ML/AI algorithm allegedly can match the same visual image ignoring crops/distortions/etc.
TW: Apple, CSAM
ML/AI classifiers have issues with recall and precision on matching images and text. Think about the hot dog app! You can find plenty of papers on tricking ML/AI to think two images have the same things when they're clearly different to the human eye.
ML/AI classifiers have issues with recall and precision on matching images and text. Think about the hot dog app! You can find plenty of papers on tricking ML/AI to think two images have the same things when they're clearly different to the human eye.
TW: Apple, CSAM
We already struggle with matching images correctly. But now Apple is asking us to accept an algorithm we have no insight into to compare photos on our devices against the NCMEC CSAM corpus.
Positive matches could be someone in that corpus just many years apart.
We already struggle with matching images correctly. But now Apple is asking us to accept an algorithm we have no insight into to compare photos on our devices against the NCMEC CSAM corpus.
Positive matches could be someone in that corpus just many years apart.
TW: Apple, CSAM
This penalises survivors for surviving. What does Apple and NCMEC intend to do for survivors who positively flag the algorithm? What will they do for survivors re-traumatised by this experience of losing access to their personal device and data?
This penalises survivors for surviving. What does Apple and NCMEC intend to do for survivors who positively flag the algorithm? What will they do for survivors re-traumatised by this experience of losing access to their personal device and data?
TW: Apple, CSAM
A few quick updates to this thread. I read through Apples updates and supporting docs (we're still in a blackbox).
I'm aware they're thankfully not using a ML classifier. Yes it's still an issue. Once again, this system should not exist.
A few quick updates to this thread. I read through Apples updates and supporting docs (we're still in a blackbox).
I'm aware they're thankfully not using a ML classifier. Yes it's still an issue. Once again, this system should not exist.
https://twitter.com/_llzes/status/1423836851555557377
TW: Apple, CSAM
@matthew_d_green has written several excellent threads on this issue and I highly recommend following him to stay up to date. For folks unaware of how hashes and collisions work, he explained in this thread:
@matthew_d_green has written several excellent threads on this issue and I highly recommend following him to stay up to date. For folks unaware of how hashes and collisions work, he explained in this thread:
https://twitter.com/matthew_d_green/status/1423084244952289281
TW: Apple, CSAM
I want to briefly step away from the technical issues of Apples system and talk about the hypothetical: what if it worked perfectly as intended? NCMEC only receives true positive reports. What now?
For the vast majority of victims and survivors: nothing.
I want to briefly step away from the technical issues of Apples system and talk about the hypothetical: what if it worked perfectly as intended? NCMEC only receives true positive reports. What now?
For the vast majority of victims and survivors: nothing.
TW: Apple, CSAM
NCMEC's By The Numbers state:
- "The CyberTipline has received over 82 million reports"
- "CVIP has reviewed over 322 million images/videos"
- "Over 19,100 victims have been identified by Law Enforcement"
CVIP began in 2002. So 19 years.
missingkids.org/theissues/csam
NCMEC's By The Numbers state:
- "The CyberTipline has received over 82 million reports"
- "CVIP has reviewed over 322 million images/videos"
- "Over 19,100 victims have been identified by Law Enforcement"
CVIP began in 2002. So 19 years.
missingkids.org/theissues/csam
TW: Apple, CSAM
In 19 years, the CVIP on average has identified just over 1000 victims per year.
In 2020 alone CyberTipline received 21,751,085 reports, a 28% increase from 2019.
For every identified victim you have 21,751 reports.
missingkids.org/gethelpnow/cyb…
In 19 years, the CVIP on average has identified just over 1000 victims per year.
In 2020 alone CyberTipline received 21,751,085 reports, a 28% increase from 2019.
For every identified victim you have 21,751 reports.
missingkids.org/gethelpnow/cyb…
TW: Apple, CSAM
Law enforcement do not protect survivors, and most reports and investigations are dismissed or closed. There is no universal healthcare in the US to pay for the lifelong costs that comes with trauma.
Law enforcement do not protect survivors, and most reports and investigations are dismissed or closed. There is no universal healthcare in the US to pay for the lifelong costs that comes with trauma.
TW: Apple, CSAM
NCMEC has the audacity to tell survivors they'll help "locate an attorney to help you understand your legal rights and pursue monetary restitution". Because we have no universal healthcare. Also NCMEC is not paying for the attorney.
missingkids.org/content/dam/mi…
NCMEC has the audacity to tell survivors they'll help "locate an attorney to help you understand your legal rights and pursue monetary restitution". Because we have no universal healthcare. Also NCMEC is not paying for the attorney.
missingkids.org/content/dam/mi…
TW: Apple, CSAM
Not every issue is a technical issue with a technical solution. Apple's tool does not help with the systemic issues involved, creates new harms and challenges especially for survivors of CSAM, and sets precedent for policymakers to push on more surveillance.
Not every issue is a technical issue with a technical solution. Apple's tool does not help with the systemic issues involved, creates new harms and challenges especially for survivors of CSAM, and sets precedent for policymakers to push on more surveillance.
TW: Apple, CSAM
I needed a break, rapidly adding more info + context for folks.
@SarahJamieLewis wrote a great intro thread on hashes and collisions to explain why Apple's perceptual hash for CSAM is so problematic. Read up + down thread from here:
I needed a break, rapidly adding more info + context for folks.
@SarahJamieLewis wrote a great intro thread on hashes and collisions to explain why Apple's perceptual hash for CSAM is so problematic. Read up + down thread from here:
https://twitter.com/SarahJamieLewis/status/1424261107154255874
TW: Apple, CSAM
If you're not following @Riana_Crypto you best be following one of the most brilliant people in the world by the time you finish reading this :)
Apple's tool harms all of us. I turned off my auto updates. That's a security nightmare.
If you're not following @Riana_Crypto you best be following one of the most brilliant people in the world by the time you finish reading this :)
Apple's tool harms all of us. I turned off my auto updates. That's a security nightmare.
https://twitter.com/Riana_Crypto/status/1424096252078137344?s=20
TW: Apple, CSAM
Most people cannot roll their own security. Even security people can't.
Survivors of CSAM especially have more involved threat models that require stronger security posturing. Things like having auto updates on are best practices because it's peace of mind.
Most people cannot roll their own security. Even security people can't.
Survivors of CSAM especially have more involved threat models that require stronger security posturing. Things like having auto updates on are best practices because it's peace of mind.
TW: Apple, CSAM
With Apple's perceptual hashing tool likely for survivors of CSAM in the NCMEC corpus to create collisions with and meet the safety threshold, that risk for many will result in surviors choosing to avoid security updates moving forward.
That's terrifying.
With Apple's perceptual hashing tool likely for survivors of CSAM in the NCMEC corpus to create collisions with and meet the safety threshold, that risk for many will result in surviors choosing to avoid security updates moving forward.
That's terrifying.
TW: Apple, CSAM
Survivors of CSAM deserve security updates, cloud backups of their photos and data. They should not be subject to further potential abuse and harassment because of an unsecured device that was entirely preventable by a security update.
This is preventable.
Survivors of CSAM deserve security updates, cloud backups of their photos and data. They should not be subject to further potential abuse and harassment because of an unsecured device that was entirely preventable by a security update.
This is preventable.
TW: Apple, CSAM
To share a bit of my life right now, outside of privacy and security, I spend a majority of my day job working on T&S and ML/AI safety involving CSAM. I disagree with @alexstamos on a lot of his thread but this is a good example breakdown.
To share a bit of my life right now, outside of privacy and security, I spend a majority of my day job working on T&S and ML/AI safety involving CSAM. I disagree with @alexstamos on a lot of his thread but this is a good example breakdown.
https://twitter.com/alexstamos/status/1424054571479887874?s=20
TW: Apple, CSAM
I want to specifically point out positively from @alexstamos's tweet/thread that there are mitigations Apple could create today such as a robust reporting feature. And breaking down the issues into the specific categories of CSAM is necessary to combat them.
I want to specifically point out positively from @alexstamos's tweet/thread that there are mitigations Apple could create today such as a robust reporting feature. And breaking down the issues into the specific categories of CSAM is necessary to combat them.
TW: Apple, CSAM
Lets also talk about NCMEC and Thorn's response to all of this, and why it's so harmful to survivors. Calling people who are criticising the potential harms of Apple's tool the "screeching voices of the minority" is damaging full stop.
Lets also talk about NCMEC and Thorn's response to all of this, and why it's so harmful to survivors. Calling people who are criticising the potential harms of Apple's tool the "screeching voices of the minority" is damaging full stop.
https://twitter.com/mattblaze/status/1424463091656429569
TW: Apple, CSAM
This gaslights survivors, especially those who have dedicated their adult lives to solving these challenges. This also places the onus on survivors to potentially feel the necessity to out their statuses of being victims/survivors.
This gaslights survivors, especially those who have dedicated their adult lives to solving these challenges. This also places the onus on survivors to potentially feel the necessity to out their statuses of being victims/survivors.
https://twitter.com/_llzes/status/1424483900533837827?s=20
TW: Apple, CSAM
Survivors of CSAM, especially those in NCMEC's corpus, should not have to re-traumatise themselves to "prove" that they themselves are victims/survivors that Apple/NCMEC/Thorn are claiming to be protecting but are not listening to.
They're not listening anyways.
Survivors of CSAM, especially those in NCMEC's corpus, should not have to re-traumatise themselves to "prove" that they themselves are victims/survivors that Apple/NCMEC/Thorn are claiming to be protecting but are not listening to.
They're not listening anyways.
TW: Apple, CSAM
Here's what I would like to see:
- Apple actually joins the rooms with folks working on these challenges including survivors of CSAM that are criticising the current tools they've announced
- NCMEC and Thorn work on direct support and assistance with survivors
1/
Here's what I would like to see:
- Apple actually joins the rooms with folks working on these challenges including survivors of CSAM that are criticising the current tools they've announced
- NCMEC and Thorn work on direct support and assistance with survivors
1/
TW: Apple, CSAM
- NCMEC and Thorn have both created several tools to report or identify, and collected collectively decades of data. Conduct follow up surveys on outcomes of survivors to direct psychological, medical, housing, financial support for survivors in the long-term
2/
- NCMEC and Thorn have both created several tools to report or identify, and collected collectively decades of data. Conduct follow up surveys on outcomes of survivors to direct psychological, medical, housing, financial support for survivors in the long-term
2/
TW: Apple, CSAM
- Apple doesn't destroy years of security industry outreach to get the most vulnerable to update their devices and undo this tool. Create a robust reporting feature and other mitigations in the meantime. We know ML/AI is not there yet. This isn't it either.
3/
- Apple doesn't destroy years of security industry outreach to get the most vulnerable to update their devices and undo this tool. Create a robust reporting feature and other mitigations in the meantime. We know ML/AI is not there yet. This isn't it either.
3/
TW: Apple, CSAM
- Survivors of CSAM should not out themselves as victims/survivors because of the harmful language NCMEC and Thorn used in their PR statements. NCMEC and Thorn should apologise, especially to survivors, and work on their language moving forward.
4/
- Survivors of CSAM should not out themselves as victims/survivors because of the harmful language NCMEC and Thorn used in their PR statements. NCMEC and Thorn should apologise, especially to survivors, and work on their language moving forward.
4/
TW: Apple, CSAM
- The collective desensitisation to CSAM due to how this all came out is horrifying. There needs to be more emphasis placed on warning people in advanced on the subject matter, using more accurate terminology, correcting people using inaccurate language usage.
5/
- The collective desensitisation to CSAM due to how this all came out is horrifying. There needs to be more emphasis placed on warning people in advanced on the subject matter, using more accurate terminology, correcting people using inaccurate language usage.
5/
TW: Apple, CSAM
- Dedicate resources towards tools for the autonomy of survivors. How did we land here with letting Apple dedicate the NCMEC corpus to train tools with the potential for real time real life harms against survivors for LE? Whose data is being used? Survivors.
/6
- Dedicate resources towards tools for the autonomy of survivors. How did we land here with letting Apple dedicate the NCMEC corpus to train tools with the potential for real time real life harms against survivors for LE? Whose data is being used? Survivors.
/6
TW: Apple, CSAM
Going to stop to take another break.
But I don't really get a break from this. I have to think of this every day. At work on ML/AI safety on CSAM and combating unintended side effects of classifiers. Working on our abusability testing framework at @PlatformAbuse
Going to stop to take another break.
But I don't really get a break from this. I have to think of this every day. At work on ML/AI safety on CSAM and combating unintended side effects of classifiers. Working on our abusability testing framework at @PlatformAbuse
• • •
Missing some Tweet in this thread? You can try to
force a refresh
