In tech firms, from what I've heard, it's rare for very early stage startups to have many data scientists. You need some product guys, hackers, and sales/marketing people. Make something, try to sell it, pivot if it fails, repeat until you succeed or run out of money
At this stage, I guess data science isn't needed because success or failure is obvious. Or sufficiently obvious not to need p-values. You have users, or you don't. You also realistically don't have big enough N's to actually run experiments even if you wanted to
Data scientists seem to enter later when firms are more mature. Changes are more incremental, and they're run through A/B tests, datasci folks pore over metrics, before deciding whether or not to shift. I think this is an intrinsically slower process
It's more focused on incremental changes and just isn't a great fit for large "regime shift" or "new product" changes
Now the actual point of the thread: let's think about how this applies to public policy and regulation. There are many policy-evaluation data folks (many empirical economists!) who play a "data science" like role for policy evaluation
But I wonder what's the equivalent of the startup CEO/hacker-team for early-stage policy product development? By analogy, you'd want a small team of "builders", taking risks, very sensitive to on-the-ground feedback and early success/failures, willing to pivot quickly
Perhaps paying less attention to p-values. Again, at this stage, success or failure of a _tech product_ is obvious and not really a matter of statistics*
*Policies have hidden consequences! So things of course may not be as simple as "hey we make $$$ now"
*Policies have hidden consequences! So things of course may not be as simple as "hey we make $$$ now"
There doesn't seem to be an academic economics subfield which maps super cleanly into this kind of role. Again, possibly imperfect analogy, but just a thought. Might be interesting to think what an academic subfield centered around doing these kinds of things would look like
• • •
Missing some Tweet in this thread? You can try to
force a refresh
