The @uscensusbureau’s has started releasing results from its new, weekly, Pulse survey. It includes some great data, but we need to be careful in how we use & interpret the results. Here’s a thread (with thanks to my @urbaninstitute colleagues @_Rob_Santos & Doug Wissoker)
Some background: The Pulse survey “is designed to quickly and efficiently deploy data collected on how people’s lives have been impacted by the COVID-19 pandemic.” Data collection began 4/23/2020 and is expected to last for 90 days. | census.gov/programs-surve…
The survey includes information about children’s education from home; internet and computer access; employment and expected income loss; food sufficiency; mental health metrics; and housing owner/rental payments.
The data are weighted to *all adults in households*. This may cause some issues in understanding summary statistics for experiences of households and children. Ideally, for those kinds of analyses, these would be conducted at the household and child level, respectively.
While the survey itself is a survey of households, the weights are designed to make inferences about adults living in those households, not necessarily children living in those households.
Let’s look at some examples. For loss in employment income, the Census definition is, “Percent of adults in households where someone had a loss in employment income since March 13, 2020.” We can break this down by race and other characteristics. 
This is NOT saying that 60% of Latinos—or even 60% of Latino households—experience loss of employment income since March 13. It IS saying that 60% of Latino adults live in a household with at least one adult who experienced employment income loss.
In, say, a Latino HH with 3 working adults, if one adult experienced income loss, the Census weights would count all 3 adults as living in a household in which someone sustained an income loss. This is correct, but may not be what your audience thinks that they are seeing.
If we want to understand what’s going on at the family or HH level, this is problematic. If we want to understand what’s happening with adults living in the family or HH, this would be fine.
The simple description of the results are probably not sufficient. It is different than an estimate that counts the income loss of each adult. It’s also not the same as saying the percent of HHs where at least one person experienced income loss.
Let’s look at education data--“had classes cancelled.” These say that 45% of Latino *adults in HHs w/children* had their children’s classes cancelled. It is not saying that 45% of Latino children had their classes cancelled. It’s a different interpretation.
How about trouble paying rent or food insufficiency? Here, it does make sense to think about the share of adults who live in a HH where there was trouble paying rent or did not have enough food. These are HH standards of living and the standard interpretation would be correct.
To sum up, be careful about interpreting some of the Pulse survey results:
-For HHs, not every metric represents a HH-level measure. Interpret carefully.
-For individuals (adult or child). Interpret VERY carefully.
-For survey respondent (micro data). Good to go.
-For HHs, not every metric represents a HH-level measure. Interpret carefully.
-For individuals (adult or child). Interpret VERY carefully.
-For survey respondent (micro data). Good to go.
The Pulse survey (small business one too) is a terrific resource and great data set to give us an insight into Americans’ experiences during the COVID pandemic, but we just need to be careful in how we use and interpret the results. /end thread
