#JSM2018 Next speaker is John Eltinge from @uscensusbureau. Transparency when integrating multiple data sources - adding admin data to survey data; starting with admin data and adding survey data to make up for limitations in admin data 
#JSM2018 Eltinge Transparency, Reproducibility and Replicability (TRR) - Transparency definition - sufficient info to evaluate quality/risk/cost/value profiles of products, implications of these for analyses, and improve processes
#JSM2018 Eltinge Reproducibility = Same results from different analyst using same dataset and methods. Replicability = Same inferences from Same population with new data collection
#JSM2018 Eltinge Sample surveys pretty luxurious in control we have over data collection process. What are “best practices “ for admin, organic data?
#JSM2018 Eltinge Need to align operational definitions of transparency, reproducibility, and Replicability with mission statements of federal statistical agencies
#JSM2018 Eltinge What is actually meant by timeliness, other quality dimensions? Accuracy, especially in admin/organic data? If wanted to be fully transparent, would spend all time documenting every step to extra detail. How much documentation is needed to be transparent?
#JSM2018 Eltinge Which features are important to report? Super important in field decisions for adaptive or responsive designs. Field staff has traditionally been making these decisions without documenting them.
#JSM2018 Eltinge What metadata is needed with multiple data sources? How will these be integrated? What is needed to document this? Example: model development in small area estimation can have lots of steps, sensitivity to models
#JSM2018 Eltinge Anchor transparency, Replicability, reproducibility in mission statement of agency. Communicate that with improvement is goal, not perfection.
