Thread by @TaraSinc

Today I shared some thoughts at the #FedFMC session on #data regulation. Understanding the potential benefits and challenges of new sources of data is an important step to getting regulation right. frbatlanta.org/news/conferenc…

I focused my comments at the #FedFMC on my experience working with private data to develop public insights: opportunities to use new sources of data to learn more about the world around us, e.g. economic indicators and societal trends. Like what we do at the @indeed #HiringLab.

Private companies today are collecting vast amounts of information on everything from what we look for in goods and services, houses and spouses, to how we spend our time and money when we think nobody’s looking.

Every action in any way connected to the internet becomes #data in the hands of private companies.

Every keystroke, every click, every hover, every scroll.

All this data is used for improving business services and products and developing new ones, as well as marketing and advertising. But, with the right incentives, it also has great promise to improve our understanding of the world around us.

Statistical agencies want to use private data to improve and expand the statistics they provide, and the press, other companies, and the stock market are all ears for insights shared by companies from their new sources of data.

When economists and other social scientists first started working with Big Data several years ago now, we thought we’d find the Holy Grail. And the promises are still quite exciting.

(In fact, I was part of a panel at SXSW in 2016 entitled “How Data Science Can Help Avoid the Next Recession” so we were really excited about the potential of this data) schedule.sxsw.com/2016/events/ev…

(I even wrote a project syndicate op ed on how excited I was about the data)
project-syndicate.org/commentary/big…

One of my favorite promises of private #data:
Big data = less talk, more actions
Private data often relies on observing actions rather than survey responses. This is particularly important given research on challenges in conducting surveys today, e.g. mitpressjournals.org/doi/abs/10.116…

I’m still excited about all the potential in private data, but we’ve also learned about a lot of perils. As we’ve dug into the data, we’ve learned it’s less Indiana Jones, and more real life archaeology.

A key challenge: businesses need the right incentives to collect and share data that could have public benefit. Wrong incentives could leave the data that could prevent the next recession locked away and forgotten in a server warehouse somewhere.
giant.gfycat.com/WealthyImagina…

It’s also important to remember that it’s hard to separate shifts in market share from shifts in the economy or society as a whole. We need to be very careful not to assume that data being collected by a private company is a representative sample of the population.

I concluded my remarks by emphasizing that not all #data are created equal. There’s lots of different kinds of data being used in lots of different kinds of ways. “Nuance” was a big theme of our session, and we need to understand the nuance to get regulation right.

And any tweet thread on this topic should emphasize the continued importance of trusted government #data. New data sources are a complement, not a substitute, to the hero’s work being done by statistical agencies. FIN

PS: We got great questions from the audience for the Q&A. I'll highlight two here.

#1: Yes! We have to think really carefully about recency bias in private data! There are techniques to address it, but companies need the right incentives to take bias seriously.

#2: To be clear, I only claimed that private data might be useful in the future. Many are already trying to use it, and I have big dreams, but it's still early days. I did get to pitch my work with @marthagimbel on labor market mismatch here, though :-) hiringlab.org/2018/09/20/us-…

Now to end this tweet thread for real - although it hasn't been a hardship to tweet while being treated to this view.