Thread by @bencasselman

Anyone interested in how economists weigh evidence and debate conclusions should see the back-and-forth about teen pregnancy in today's @nberpubs. (brief thread to follow)

This story starts in 2014 with @kearney_melissa & @phil_wellesley's working paper (later published in AER) finding that the MTV show "16 and Pregnant" may have reduced teen pregnancy rates.
nber.org/papers/w19795

As you might expect given the topic/findings, that paper generated a *lot* of media coverage. It also got lots of attention in econ circles for its creative methodology.

Today, @DavidAJaeger & coauthors published their own @nberpubs
working paper arguing Kearney et al's methodology was flawed. (This is actually the latest salvo in a longer-running back-and-forth on this issue.)
nber.org/papers/w24856

Kearney & Levine have published their own response defending their methodology: nber.org/data-appendix/…

And @ariellakl & Kevin Lang have their *own* working paper looking at the larger methodological lessons of the debate.
nber.org/papers/w24857?…

I'm not going to summarize the details of this dispute on Twitter, let alone try to adjudicate it. The essence of it comes down to how to evaluate whether what's billed as a "natural experiment" is truly a proper experiment.

But I highlight this exchange because it's a great example of the way economics really works, in a way the public rarely sees.

Someone came up with a creative way to explore an important, policy-relevant question, and reached a striking conclusion. Someone else raised substantive issues with that finding. There was, and will continue to be, arguments and counter-arguments.

The pat conclusion to this is, "And eventually the science will settle and we'll know who's right." Nah, probably not. I doubt we'll reach a satisfying, "now we all agree" consensus.
Maybe you find that frustrating. I don't.

This kind of exchange is the best case for econ-as-science. Science, with all its incumbent uncertainty and false leads and muddy conclusions. (As @cragcrest so beautifully described it:
fivethirtyeight.com/features/scien… )

I don't mean to wax romantic here. I'm sure there are some hard feelings in this dispute. Maybe not everyone is operating in good faith. Certainly the incentives in academic econ can be problematic,

But at a time when the public discourse around econ often boils down to, "ha ha! you didn't see the recession coming!" it's useful to be reminded how econ-as-a-science actually works. </fin overly long thread>