Thread by @Edu_Historian

Education "ratings" are inherently biased against schools serving racially minoritized and economically marginalized students.

A brief thread...

I am, right now, looking at the results of a linear regression analysis of "percentage of students who are 'at-risk'" on "metric score." (For one large urban district)

Translation: what is the correlation between demography and the components of the school rating formula?

I'll keep the district anonymous, since I'm not sure if this analysis is public.

But it's fairly typical in the way it measures school quality. (Which is to say: very narrowly, emphasizing test scores, and ignoring most of what we care about in education.)

Here are the STAGGERING results...

The correlation between the percentage of "at-risk" students and standardized test score "proficiency" (high school) is 0.8

That's...massive. That's like the correlation between rain and rainclouds.

Meanwhile...

The correlation between the % of at-risk students in a school and that school's GROWTH RATE on standardized test scores is...like...zero.

Wow.

OK not wow. Because...well...that's exactly the point of growth scores. They are designed to take into account the uneven distribution of students across schools.

Schools serving privileged kids are not "better" just because they have higher test scores. Those students would score roughly the same no matter where they went to school.

We can't say, just by looking at proficiency rates, what the quality of a school is. Because...demography.

*Now I'm cheating a bit, I'll admit, because I started with HS proficiency rates and then I gave you middle school growth scores, since they don't have growth scores for high school. But...well...growth scores are designed NOT to correlate with demography. So, forgive me! Onward.

And what are the other components of their rating system?

Like...what are all the things we would want to measure other than test score proficiency rates and test score growth rates???

Go on. Weigh in. I'll wait.

OK I lied. I'm not waiting.

Because you KNOW that they aren't measuring things like how safe kids feel, what the instructional climate is like, how diverse the school is, how much access to arts and music education they get, etc. Nope.

Attendance. Attendance (yes that's right, two attendance measures). Attendance growth (3!).

Graduation. Graduation.

And some AP/IB rates.

Attendance correlates with school-level % of at-risk students, though only moderately. (The attendance growth score is near zero BTW, as it should be.)

Graduation correlates, too...moderately.

AP/IB participation correlates strongly.

So...what are we measuring here?

When we look at where students started and where they finished, we see the ground they covered at highly-rated schools was basically the same as the ground they covered at low-rated schools.

[Clears throat]

Now, people. Do we need to use the "starting line" metaphor here???

Don't make me do that.

So we're essentially saying: "Hey, school: You serve a bunch of the kids we need to do right by in our society. And it seems you're doing about the same for those kids as your fancier counterparts are doing for their more privileged populations. But..."

"Buuuuuuuuuuuuuuut...."

"But guess what?"

"You get an F."

"Sorry."

It actually isn't an F.

They do stars.

"No stars for you."

I shouldn't joke because this is a moral crime.

These low ratings will then be used in a variety of ways, including by racially and economically privileged parents looking for schools.

As a result, they will exacerbate segregation.

If our measures were perfect, we would still need to exercise caution with regard to the possible consequence of exacerbated school segregation.

But our measures are the opposite of perfect. Our measures are an absolute dumpster fire.

Take all the school ratings -- those created by districts, those created by states, those created by unregulated and unaccountable 3rd parties like GreatSchools.org, etc.

Take them all. Put them in a box. Weigh it down with bricks. And drop it to the bottom of the ocean.

Thank you for coming to my TED talk.

I will be signing autographs in the hallway.

And by signing autographs, I mean I will be swearing under my breath for the next five hours about the neoliberal obsession with rating schools and the harm that it does to educators, kids, and communities.

If you want the report I was referencing, give me 24 hours to figure out if I'm allowed to share.

Meanwhile, here's a study my team did, where we found similar trends (see, particularly, Table 4): journals.sagepub.com/doi/full/10.11…

Update: here's the report: sboe.dc.gov/sites/default/…