As promised: Why we DON’T need ANOTHER South African Covid dashboard that merely re-presents the data in the public domain. (A thread 1/26)
There is a sudden proliferation of public-facing dashboard websites that seek to present data on Covid in South Africa. What do these do; what data do they present; and what insights and value-add can they bring? (2/26)
In general, using a variety of web-apps (shiny etc), these dashboards provide ‘live’, ‘real-time’ ‘data’ on Covid, using dataviz techniques. (3/26)
In fact, in many cases, the emphasis would seem to be on the ‘shininess’ of the presentation, without much regard for the quality or utility of the data underpinning the dataviz. (4/26)
It would seem that in rushing to bring their dashboards live (more later), few of the producers thought it necessary to consult with experts to help interpret what is being presented. (5/26)
Why does this matter? Because the data provided in the nightly releases from the Minister of Health are largely UNinterpretable. Let me explain – a sub-thread.(6/26)
a. The number of tests reported each day are classified by when the results are _presented_ to DoH. They do not refer to the tests _conducted_ in the previous 24 hours.
b. The tests reported on day x may have been collected several days previously. (7/26)
b. The tests reported on day x may have been collected several days previously. (7/26)
c. Likewise the positive test results may take longer to come through than negative test results. This means that neither the timing, nor the positive-test prevalence can reliably be calculated from the data provided. (8/26)
d. The daily release does not record the number of tests by province. The NUMBER of positive cases is highest in Gauteng. Is the proportion of tests returned positive higher in GT? Slow down! It is impossible to know _from the public data_. (9/26)
e. Some dashboards flag the date of lockdown. This is only interesting in terms of the cases we have seen in the last 10-14 days. The lockdown would not have had a direct impact on those testing positive immediately after the lockdown, given the known incubation period. (10/26)
f. The testing is NOT random. And the protocols for testing have changed almost continually since the start. So simply representing positive cases as cases per 1000 in a given region / province is (as badly) misleading. (11/26)
g. The distribution – spatially, and by demographic characteristics, not to mention volume, and by source (public/private) – of tests conducted would appear to be changing rapidly, and in non-stable ways. (12/26)
h. Even the recently introduced programme of community screening and testing is NOT random. Those who are sent for testing have been *screened* for being at risk of being infected. This cannot give a measure of population prevalence. (13/26)
i. On top of that, NOT ALL infections are being picked up. Possibly 70% of infections are asymptomatic but contagious. And even those with mild symptoms may not be identified. The number of cases reported is probably underestimated by about 10 times. (14/26)
j. The data on deaths are curious. More positive cases in GT; more deaths in WC. What’s going on? (15/26)
k. The data on deaths reflect only direct-direct deaths (ie. deaths from Covid, reported as Covid). Deaths from Covid, attributed to other sources are not being reported as such. (16/26)
l. And there are the deaths that are NOT Covid-related that will emerge as the health system gets overloaded. And –finally- the deaths that will NOT occur from other causes in the near future as a result of premature Covid-mortality. (17/26)
m. This is a global phenomenon (see the NYTimes piece from yesterday: nyti.ms/34QerxA). In South Africa, the SAMRC has made public its recent near-real-time mortality monitoring system (samrc.ac.za/reports/report…). (18/26)
n. (On this, see the excellent thread from @AndrewNoymer ) (19/26)
o. This system will allow us to track the deviation of deaths from a historical, ‘normal’ pattern. But the deaths REPORTED bear little or no resemblance to ACTUAL mortality. (end of subthread) (20/26)
And what you CANNOT do, is try to extrapolate from whatever trend you currently see (see above; changing testing protocols; new screening and testing programmes). So please: DON’T. (22/26)
Is this re-presentation of public information useful? Yes and No. Yes, because people may see the data that they might otherwise miss. AND THAT’S IT. But against that; the data are NOT WHAT THEY SEEM. (23/26)
They certainly do NOT inform expert opinion, or the response from those tasked with responding. So what this proliferation is really about is this: producers driving eyeballs to websites; and pretending that they are experts (cf Zapiro: zapiro.com/200419dm). (24/26)
Could government be doing more to place more useful information in the public domain? Yes! (but this should not be allowed to detract from the urgency of responding to the epidemic; and it will be less ‘timely’ than is presented at the moment. (25/26)
In other words, you, the viewer, are the product. Not the data. Caveat lector! Fin (26/26)
