Thread by @rnishimura

Live tweeting the #AAPOR session The Panel on the Panel: Development and Testing of a Probability-Based, Nationally-Representative Survey Panel for Federal Use

First is Victoria Dounoucus (RTI) presenting Qualitative Work to Inform Contact Materials and Baseline Questions for the Ask U.S. Panel Pilot

Cognitive interview in Microsoft Teams for ~1 hour, with 30 interview (21 in English, 9 in Spanish)

First they tested the logo.
Winner logo was #4:
They like the US map and how the check mark indicated action

Asked their understanding of "panel"
They understood was a selected sample for a survey, but did not mention participating in the survey over repeated times

When asked again after receiving some information about the panel the understanding continues the same.
Action: Including the expression "over time" in the invitation letter to clarify that.

Next presenter is Dustin Williams (RTI) presenting Methods and Experimental Evaluation of Strategies for Online Panel Recruitment

Respondents with no internet were provided a tablet with a paid internet plan.
4 of 53 respondents that were provided a table completed the topical survey and none completed the final survey 😱😅

Response Rates

Next is Amanda Smith (RTI) talking about Just Scan It: The Effectiveness of Including QR Codes in Panel Recruitment Materials

Why QR codes?
Reduce burden and mitigate error in typing URLs

Pre-COVID: QR codes were not very often used
Post-COVID: QR are everywhere! (Seriously! Can we just stop using QR codes for menu restaurants? 😛

Two mechanisms to acess web survey:
1) QR code
2) URL

Note: taking this pic was a bit difficult because my phone kept trying to read the QR code 😂

Much higher use of QR code than previous study!

Half of mobile respondents used QR code and the majority of them are smartphones

Characteristics by model of access

Next is Carlos Macuada (RTI) talking about Tailoring Materials to Respondents: Heterogeneous Treatment Effects in Survey Experiments

To tailor the survey materials we need to have a good understanding of the population.

Are there differences across different subgroups with respect to certain design features?
Used BART to answer this question.

Results:

Next and last presenter of the session is Emilia Peytcheva (RTI) presenting Evaluation of Nonresponse Bias in the Ask U.S. Panel Pilot

Focus on nonresponse bias at receuitment
Two-phase design with subsampling for nonresponse:
* Face-to-face
* Additional $5 incentive
* Provide tables

Weighted screening response rates:
Phase 1: 14%
Overall (Phase 1 + Phase 2): 37%

Examined 298 estimates
14% of them found significant "bias"

Results:

That's a wrap! Great session about how Census is working in building a national panel!
/TheEnd

@threadreaderapp compile unroll thread