,
34 tweets,
13 min read
Read on Twitter
I’ve been trying to stay off Twitter to focus on writing and my toddler these days, but today people had things to say on the internet about charter schools.
@jonathanchait
nymag.com/intelligencer/…
@BenMathisLilley
slate.com/news-and-polit…
1/N
@jonathanchait
nymag.com/intelligencer/…
@BenMathisLilley
slate.com/news-and-polit…
1/N
Now I finally understand what people mean when they say “RIP my mentions.”
2/N
2/N
tl,dr: Shocker -- the academic is here to say that neither article gets it exactly right, and that when it comes to charter schools, a lot is "it depends."
3/N
3/N
They are focusing on a piece by @dynarski and me from 2016 brookings.edu/research/massa… which links to and cites much of our and our colleagues research on charters schools in MA to the date of publication.
4/N
4/N
Most of our work focuses on lottery-based estimates of charter school effectiveness. Why? As economists, we aim to estimate the causal effect of policies and programs.
5/N
5/N
A simple comparison between program participants and non-participants could confuse the effect of the program with being the type of person to apply to a program ("selection bias").
6/N
6/N
Controlling for baseline test scores, for example, helps mitigate the selection bias, but cannot account for selection due to "unobservables" -- things like motivation, family characteristics, etc.
7/N
7/N
Lotteries provide a natural experiment, free from selection bias. However, charters w/ lotteries are not necessarily representative of all charter schools. See this graph, from my piece in @FutureofChildren for the pros and cons of various methods for charter school eval.
8/N
8/N
Some of our work has included comparisons between lottery and observational studies, including the original article on Boston charter schools in @QJE economics.mit.edu/files/6335
(I was an RA, not an author on that one.) (In my dreams...)
9/N
(I was an RA, not an author on that one.) (In my dreams...)
9/N
We did something similar in a 2013 @BostonFoundation report. seii.mit.edu/wp-content/upl…
When we compare lottery and non-lottery/observational approaches, a clear pattern emerges. The oversubscribed schools are more effective.
11/N (missed the count in the prior tweet -- oops)
11/N (missed the count in the prior tweet -- oops)
But column 4, which pools together both oversubscribed and undersubscribed charters for non-lottery estimates shows that the effects almost exactly match the lottery-based estimates (column 1). (Repaste-ing table)
12/N
12/N
How can this be? There are so many more oversubscribed charters, that when describing the sector as a whole in Boston, the effects (by either method) are large & positive.
(Can only study closed schools by nonlotto – by def'n there is no entity to go to for lotto records)
13/N
(Can only study closed schools by nonlotto – by def'n there is no entity to go to for lotto records)
13/N
We don’t do this particular lottery/non-lottery exercise in the 2016 paper referenced in the Brookings piece and by both of the recent articles. economics.mit.edu/files/9799
14/N
14/N
That’s because the article is a specific follow-up to the original @QJE piece, asking a different Q: are the schools that are successful @ raising state test scores (as shown in lottery-based analysis) also successful @ raising other outcomes like AP, SAT, & college-going?
15/N
15/N
We do take pains to refer to “oversubscribed” and “lottery-based” throughout the paper, so that it is clear we are not making claims about every charter school in Boston. We also end the paper as follows:
16/N
16/N
As for charters across MA, Angrist, Pathak, & Walters (2013) clearly show that nonurban charters produce no or negative effects on test scores. seii.mit.edu/wp-content/upl…
Urban schools, including those in Boston, but also Framingham, Cambridge, New Bedford, Chicopee, Lynn, Fitchburg, Springfield, Chelsea, and Everett (see the appendix for a list of schools).
18/N
18/N
The point @dynarski and I made about Question 2 was that the cap lift would not affect nonurban areas. If those areas wanted charter schools, they could expand under current law. However, the cap binds in urban areas, exactly the areas that have documented positive effects.
19/N
19/N
If I was ruling the world, however, I would not have suggested the cap lift as written in Question 2. Instead, I like the “smart cap,” which was the policy implemented as a previous cap lift in MA, which grants new charters to operators that are “proven providers.”
20/N
20/N
@ElizabethSetren, Chris Walters, and I study the proven provider policy in this recently updated paper, and show that the state chooses proven providers effectively, and that replication campuses are just as successful as their parent campus. seii.mit.edu/wp-content/upl…
21/N
21/N
I should note that this study covers 14 of 15 middle school charters operating in Boston at the time. That one missing charter would have to have impossibly large negative effects to turn the overall charter effectiveness to a zero.
22/N
22/N
I think that covers the issues that were brought up in the articles, but let me take a moment to highlight some new research on charter schools that has come out since the Brookings article (not meant to be comprehensive).
23/N
23/N
.@ElizabethSetren looks at the effects of attendance at a Boston charter on students with disabilities and English learners, looking both at test scores and classification
drive.google.com/file/d/16WFL5v…
24/N
drive.google.com/file/d/16WFL5v…
24/N
Julia Chabrier, Phil Oreopoulos and I summarize the research using charter school lotteries in the Journal of Economic Perspectives.
aeaweb.org/articles?id=10…
25/N
aeaweb.org/articles?id=10…
25/N
@Dynarski, Brian Jacob, @dandhubb and @sroblesc look at a non-profit charter chain in Michigan using lottery methods: edpolicy.umich.edu/files/01-2018_…
26/N
26/N
Atila Abdulkadiroglu, @metrics52, Yusuke Narita, and @daGuruG use a universal enrollment system in Denver to do some cool econometrics and estimate charter school effects: economics.mit.edu/files/13447
27/N
27/N
And I summarize it all (minus the too recent to be included MI article, and more) in my piece in the Future of Children: futureofchildren.princeton.edu/news/charter-s…
29/N
29/N
On a side note, I was thrilled to vote for @SenWarren for senator. (I ran into her on the street in Cambridge once and could not keep it together.)
30/N
30/N
I don’t live in MA any more, but if @SenWarren is the nominee, I would vote for her again, even though I disagree with her on charter schools.
If I had a chance to talk to her, I would ask her to listen to the voices of Indigenous people. Then we could talk charters.
31/31
If I had a chance to talk to her, I would ask her to listen to the voices of Indigenous people. Then we could talk charters.
31/31
32/31 Oh, and should definitely be including this recent paper by @CamilleTerrier1 and Matthew Ridley which shows that charters in MA increase PPE in sending districts as well as test scores (this is due to an unusual funding formula in MA) seii.mit.edu/wp-content/upl…
And a follow up today from me. Please excuse the threading error:
Looks like both journalists have now updated their articles reflecting some of this Twitter conversation.
