Crémieux Profile picture
Oct 4, 2024 • 10 tweets • 4 min read • Read on X
There are people who desperately want this to be untrue🧵

One example of this came up earlier this year, when a "Professor of Public Policy and Governance" accused other people of being ignorant about SAT scores because, he alleged, high schools predicted college grades better.Image
The thread in question was, ironically, full of irrelevant points that seemed intended to mislead, accompanied by very obvious statistical errors.

For example, one post in it received a Community Note for conditioning on a collider. Image
But let's ignore the obvious things. I want to focus on this one: the idea that high schools explain more of student achievement than SATs

The evidence for this? The increase in R^2 going from a model without to a model with high school fixed effects

This interpretation is bad. Image
The R^2 of the overall model did not increase because high schools are more important determinants of student achievement. This result cannot be interpreted to mean that your zip code is more important than your gumption and effort in school.

If we open the report, we see this:Image
Students from elite high schools and from disadvantaged ones receive similar results when it comes to SATs predicting achievement. If high schools really explained a lot, this wouldn't be the case.

What we're seeing is a case where R^2 was misinterpreted.
The reason the model R^2 blew up was because there's a fixed effect for every high school mentioned in this national-level dataset

That means that all the little differences between high schools are controlled—a lot of variation!—so the model is overfit, explaining the high R^2
This professor should've known better for many reasons.

For example, we know there's more variation between classrooms than between school districts when it comes to student achievement.

As another example, we know that achievement gaps exist along the whole continuum of school and district quality.

If the issue was really zip codes, high schools, and so on, this shouldn't be the case.

The other thing this professor should've known is that high school is biased! GPAs are biased too!

The bias in GPAs has actually been exploited: elite high schools inflate grades and don't report class ranks, so students appear better than they are. Image
But you know what isn't a biased tool for admissions? Just one thing: test scores.

Want to learn more? Here are some sources:

x.com/cremieuxrecuei…

cremieux.xyz/p/what-happens…

cremieux.xyz/p/bias-in-admi…

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Crémieux

Crémieux Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @cremieuxrecueil

Feb 23
This study is being investigated since it includes results by convicted fraud Stephen Breuning.

Without his huge, fake estimates, the meta-analysis is riddled with publication bias. Correcting for it makes the meta-analytic estimate practically and statistically nonsignificant. Image
It is also just unserious to think that a meta-analysis including obvious rubbish should overturn much better established facts.

For example, one of the cited studies claimed to show IQ scores improving by 3.64 g (55 IQ points) when kids (n = 10) were offered a $5 cash prize.
You reveal a lot about yourself if you take nonsensical and unreplicable results seriously.

This meta-analysis never should have been published because of the included fraudulent work, the included garbage work, and the failure to consider psychometric bias explaining results.
Read 8 tweets
Feb 21
People across the political aisle engage in conspiracy theorizing at markedly similar rates, just about different things. Image
Q: Does each side do this to the same extent?

A: Probably not! In the case above, to get the appearance of total symmetry, you have to include a lot of different conspiracies that are very Trump-related.
Q: What about general conspiracist intent and ideation?

A: That's plausibly higher on the right in the U.S., even after accounting for measurement non-invariance. It's not globally higher, but few correlates of politics are globally consistent. More on this soon.
Read 11 tweets
Feb 20
The biggest news today should probably be about one of the Executive Orders from yesterday evening.

Trust me, it's big.

The President just authorized DOGE to start cutting regulations🧵 Image
This order starts off huge.

Remember those recently-created DOGE Team Leads going into every agency? They're going to work with agency heads and the OMB to review all of the regulations across a number of huge categories.

Which categories? Let's see.

Image
The first category is those rules and regulations which violate the law of the land: unlawful and unconstitutional regulations, things that agencies enacted but which they shouldn't have been able to. Image
Read 17 tweets
Feb 16
It's projected to take the City of Boston 20 years to build a single train station.

So here's a thread on how long it took to build other things.

It took six years to build the Transcontinental Railroad from Omaha to Sacramento, including hundreds of stations along the way. Image
It took 410 days to build the iconic Empire State Building. Image
After the French failure to build a canal in the Isthmus of Panama, the U.S. set out on a second attempt to connect the Pacific and the Atlantic.

It took the U.S. and Panama just eight years to build this enormous megaproject that still facilitates billions in trade each year.Image
Read 20 tweets
Feb 13
The biggest news of the day is not so much that @RobertKennedyJr was confirmed by the Senate, but what he's going to do next.

@realDonaldTrump just issued an Executive Order making it official:

America stands against chronic disease and closed science🧵Image
The first thing the EO does is outline the problem

It talks about how unhealthy America is, how unacceptable that is, and how we have a duty to change that

We do: Americans should not just be the richest people in the world, they should be the hottest, healthiest, and strongest Image
Now beyond outlining the problems America faces, the Order outlines some policy prerogatives that will be front-and-center during this new administration.

I want to preface something here: Regardless of what you think about the people involved, something here will make you happy Image
Read 13 tweets
Feb 12
The biggest news of the day should once again be about DOGE.

A new Executive Order was passed a few minutes ago.

It empowers DOGE to spearhead the complete reorganization of the federal government🧵 Image
The first part of this Order is simple:

The OMB will put out a plan to make the federal workforce smaller and more efficient, including a stipulation that agencies must remove four existing employees for each new hire, with some exceptions. Image
The second part is meatier.

New hires have to be approved by newly-installed DOGE Team Leads in each agency. These Team Leads will report what goes on in the agency they're assigned to on a monthly basis.

But that's not even the big part yet. Image
Read 10 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(