Thread by @kendraschaefer

The hits just keep on coming. On Friday, China's cyberspace watchdog (CAC) released draft rules governing "deep synthesis tech" - deepfakes and other machine-generated or -edited text, voice, video, and 3D spaces. Paraphrasing the key points below. 1/18
cac.gov.cn/2022-01/28/c_1…

This doc puts rules around how deepfake apps, question-response bots, text-to-speech generators, voice / facial / gesture manipulation, etc can be used. Will impact the creators of core tech, and platforms where such content can spread - including metaverse platforms. 2/18

The short version: Anyone providing the tools to produce synthetically-generated or -edited content must take steps to ensure the content is trackable, controllable, and legal. 3/18

In part, that means applying China's existing illegal content restrictions to media generation tech - no generating porn, politically sensitive / disallowed content, fake news, etc. 4/18

It also means keeping users informed when they are viewing content that has been synthetically created or altered, and making sure that content can be traced to its source. 5/18

Super interesting - Article 12: If you're providing the ability to allow users to mess with biometric info like faces and human voices, the person whose identity / features are being edited has to give consent for their personal info to be used. 6/18

Also interesting - Article 13: Machine-generated or edited content has to include some kind of technical marker so that the content can be traced to the source where it was generated. 7/18

Related - Article 9: If you build an app or tool that lets users generate machine-made or machine-edited content, then you must collect the real identities of those users - another way to track content back to source. 8/18

Article 14: The generated content itself must come packaged with notifications that inform users that it is synthetically generated / altered. Example: A voice file that imitates human speech must include an audio clip identifying the audio as machine-made. 9/18

Article 16: If app stores find that any app violates these rules, the app store may not list the app, or must remove the app from shelves. 10/18

Article 17: Anyone providing deep synthesis services has to be on the lookout for, and prepared to refute, false news and rumors. If such content is discovered, providers must stop its spread, and report it to the authorities. 11/18

Article 19: Anyone providing deep synthesis services has to register with the CAC within 10 days of beginning to provide the service. 12/18

Article 20: If anyone providing deep synthesis services adds new features, or launches a new product with "public opinion attributes or social mobilization capabilities," the new product has to undergo a security assessment by CAC. 13/18

My take - One interesting piece of this is that pressure is placed on multiple actors to comply - app stores, developers, platforms all have obligations here. 14/18

My other take - Of course this has implications re: censorship - these rules seek to close the door on a future where activists make videos of Xi Jinping making anti-Party statements and whatnot. But that's not the interesting bit. 15/18

The interesting bit is that China is taking aim at one of the critical threats to our society in the modern age: the erosion of trust in what we see and hear, the inability to tell what is real and to separate truth from fact. 16/18

China is able to institute these rules because it already has systems in place to control the transmission of content in online spaces, and regulatory bodies in place that enforce these rules. 17/18

So, these rules underscore the policy problem of our age: How can Western democracies fight a war against disinformation and prevent the erosion of trust and truth online, but without resorting to censorship? 18/18

Making a separate note about this one - will be a problem to implement. Current draft says tech providers have to remind content creators that creators are responsible for getting consent from person whose data is being used. Seems... wishy-washy.