,
14 tweets,
4 min read
Read on Twitter
Automating Racism: Police in China are using A.I. to mark/track ethnic minority Uighurs across the country. It’s the first known example of facial recognition being used intentionally by a government to racially profile and a massive ethical leap for A.I. nytimes.com/2019/04/14/tec…
In effect, Beijing is using powerful computer vision algorithms to sort people based on whether or not they appear to be Uighur. If they do, their information is marked and made searchable. (Below a sample of a database of Uighur faces from a paper on Uighur facial recognition). 
The Chinese startups building these systems all have some foreign investment, and are already expanding overseas. While the tech is not yet mature, it’s fast improving. It’s hard not to imagine it soon being used to track people based on race and ethnicity in other countries.
Hundreds of thousands of Uighurs have already been locked up in camps. Their home region turned into an open air prison. Here’s a summary on their treatment at home. This extends controls across China.
In the U.S. “there is most likely racism built into our algorithmic decision making, but not in an overt way like this,” EFF’s surveillance litigation director told me. “There’s not a system designed to identify someone as African-American, for example.”
In one police database, the racial profiling function sat alongside more innocuous features determining whether someone was wearing sunglasses or a mask. In a month on a handful of cameras it scanned people 500,000 times, marking them Uighur/non-Uighur.
Here’s a translation of an ad from Chinese startup CloudWalk that describes a dark use case. Basically if a Uighur has a barbecue or family visit, police are called.
In another bit of not-so subtle advertising, CloudWalk lists key control points for such tech, and shows photos of four mosques. It includes the Id Kah mosque in Kashgar, where there are more than 100 high-powered cameras and a propaganda shot of Xi Jinping sits over the Mihrab.
The same softwares are being used to track other vulnerable groups, albeit in a more targeted way.
Local police make lists of faces of the mentally ill, drug users, and petitioners. Sources said most databases are kept at a city/county level. The smaller numbers make things easier on the tech. The lists offer a glimpse of an emerging architecture of social control.
Many cities have lists of the following:
Long-term residents (w/hukou)
Temporary residents (in hotels)
Those with driver’s licenses
Those with a history of drug use
Criminals at large
People w/ criminal record
People w/ a record of drug use
The mentally ill
Petitioners
Long-term residents (w/hukou)
Temporary residents (in hotels)
Those with driver’s licenses
Those with a history of drug use
Criminals at large
People w/ criminal record
People w/ a record of drug use
The mentally ill
Petitioners
There is also a national list of Uighurs who are traveling outside Xinjiang. Which would help the systems not just determine whether someone belongs to the ethnic minority, but who they are. There are also some places with databases of foreigners.
There’s a fair bit of advertising for the next gen Chinese startups providing these services. Many have scrubbed more cavalier ads showing racial identification capabilities. I’m partial to this one from Megvii (since taken down):
For a long time democracies had a monopoly on cutting edge tech. China’s changing that. New tech is being built to authoritarian needs, often targeting vulnerable groups. It’s time for many in the U.S. to think about what constitutes unacceptable involvement in such practices.
