Thread by @rnishimura

Live tweeting session Detection and Mitigation: Developing Monitoring Tools to Ensure Quality in Interviewer-Administered Data Collection at #AAPOR

First is Joe Murphy (RTI) presenting A Dashboard for Field Survey Data Quality Management

Why an *interactive* dashboard?
Different users, different needs
Gives user control to what they need

Solution: Adaptive Total Design dashboard
* Web-based interactive design
* Both study- and IWER-level metrics
* same framework across projects

Many data sources:
Survey control system
QC system
Financial data system
Ad-hoc databases

Implemented in R Shiny
External clients connect using a web-authentification

Note to self: they have a bookmark option button on the dashboard.

Also an option for automated alerts

There is also a cross-project platform. In the future include additional metrics and apply falsification detection indicators Ina more standard way.

Next presenter is Kate Bachtell (NORC) presenting Finding Proof in Your Points: Beta Test Results for a Data Science-Driven Quality Assurance System

NORC Proofpoint: NORC quality assurance dashboard, built from SAS

Their dashboard include a real-time falsification score, which computes the probability of falsification at the interviewer-level and then aggregates up to the IWER.

Results: in SCF, 27 of 45 cases flagged as falsification were confirmed. Interviews were re-conducted.

Next speaker is Scott Glendye presenting Something’s Missing: Improvements in Interviewer Nonresponse Detection and Correction

Tools developed by Field Quality Monitoring team at the US Census Bureau to resolve interviewer issues in up to 48 hours!

Dashboard to improve item nonresponse
Looking for clusters of items that are correlated with each other

Identify missing rates in some items for interviewers to tailor recommendations and guidelines that they could use to improve that behavior

Hawthorne effects concerns -- some interviewers could be behaving in malicious ways given the monitoring of their activities

@threadreaderapp compile unroll thread