Thread by @shahitbiz

After 6+ months of work/research, we have finally launched our village/township analytics interface: shahit.biz/eng/#village (also accessible through the Tools tab)

Basically, this is a sub-database of population and detention data for XJ's 15000+ villages/neighborhoods. (1/18)

As a reference year, we chose 2018 as this was the heart of the incarceration campaign. However, since village population data isn't readily accessible, we needed to spend 1000s of hours doing busy legwork to find it, and it was usually not for 2018. (2/18)

One invaluable source was this 1987 book of village names and populations (shahit.biz/supp/villagepo…), which covers very many villages but naturally has extremely out-of-date (1986) numbers. (3/18)

We also looked for more recent data in a mix of online articles, reports, academic papers, and - last but not least - local county yearbooks. Most of these weren't from 2018 either, of course. (4/18)

So we built empirical models, matching numbers from 1986 to more recent ones for the 100s of cases where we had both for the same village. Building distributions of the average growth rate, we could then estimate 2018 pop. ranges from 1986 data with high (~90%) confidence. (5/18)

Because we had yearbooks from multiple years, we also built empirical models to predict how (reported) village populations changed over shorter periods (1-10 years), based on 1000s of observed data pairs. This let us get 2018 pop ranges from 2000-2017 (or 2019-2020) data. (6/18)

For villages/neighborhoods where we were absolutely unable to obtain anything, default high-confidence (90%) bounds based on available 2018 data were used (to give a typical interval that most villages/neighborhoods fell into). (7/18)

Analogous procedures were carried out to compute the ethnic-minority percentages, with minor differences. Likewise for the 1500+ townships/subdistricts, though these were simpler, with more data more readily available. (8/18)

To keep in line with standard statistical reporting, we chose to use permanent-resident populations (常住人口) and not hukou populations (户籍人口) - the latter typically used by police and harder to obtain. Another empirical model was built to convert where necessary. (9/18)

To estimate *minimum* detention rates, we divided the number of "detainees" for a given locale by its maximum 2018 ethnic-minority population.

"detainee" = any ethnic-minority individual detained for 15+ days (or shorter, if hurt/killed) at any point since January 2017. (10/18)

But this wasn't so simple either. Not all detainees placed in a given village/neighborhood are necessarily permanent residents there - some have their hukou there, some were only born there, some have close relatives there, etc. So we built empirical models for that too. (11/18)

Given the level of conservatism in the models and the fact that we're only using the victims we've documented, detention rates in many places are going to be quite low - or non-existent. Though there are also places where they're quite high. This will improve over time. (12/18)

Some other features:

Each village/township has its own profile page, which not only summarizes the stats but puts it on the map and lists the victims associated with that village/township. You just need to click on the name. (13/18)

Another handy tool is the search tab, which you can use to search for villages/townships by name - either in pinyin, 汉字, or the Latinized original (usually Uyghur/Kazakh) name. Can be used as a place name dictionary. (14/18)

This is still a work in progress and will be so infinitely, as we continue to populate the interface with data. At the moment, only ~50% of victims are allocated, ~51% of village names translated, ~5% of GPS coordinates set, ~63% of villages with populations researched. (15/18)

Furthermore, the analytics interface is not an end in itself, but is intended as a foundation to help dissect and destroy the XJ information vacuum. Currently, it is already being used in tandem with our victims map: shahit.biz/victimmap.php (16/18)

In the coming 3-6 months, we also plan to revive our "watchdog" function, and build a formal system for calling local village offices and police stations, the numbers for which will be stored in the individual pages. Linking victims to places makes that a lot easier. (17/18)

Finally, in the coming weeks we plan to add the first version of our global detention estimator, applying local interpolation to get a rigorous lower estimate on the global number of detainees for all of XJ. Don't expect gigantic numbers, but do stay tuned. (18/18)