Honestly kind of surprised that the @UpshotNYT thought the Stanford paper about features computers can notice in college application essays was worth amplifying (again).
A 🧵 on 3 of the very big problems with the paper.
nytimes.com/2021/05/26/ups…
A 🧵 on 3 of the very big problems with the paper.
nytimes.com/2021/05/26/ups…
Problem 1. Computers don't admit students to college; people do.
1.1 The paper shows that computers can identify correlations between essay content and income in large data sets.
1.1 The paper shows that computers can identify correlations between essay content and income in large data sets.
1.2 Humans are reading essays for insights into the author. They are not analyzing a corpus for insights into all the corpus.
1.3 A computer can identify correlations between income and style (comma use, word count) in the corpus, but can human readers do so in an essay?
1.3 A computer can identify correlations between income and style (comma use, word count) in the corpus, but can human readers do so in an essay?
Problem 2. Computers identify; humans evaluate.
Remember how people thought word clouds were super dope in 2007? And then we realized they were maybe just a gimmick, because humans don't read like computers do.
Remember how people thought word clouds were super dope in 2007? And then we realized they were maybe just a gimmick, because humans don't read like computers do.
2.1 The computers are just finding patterns, and some of them are pretty banal. Who needed to be told poor people write about being poor more than rich people do, or that rich people write more about leisure travel?
2.2 The paper does not show or even attempt to show a correlation (let alone a causal connection) between an essay's correlation with income and whether an essay's author was admitted or rejected. That's important because humans and computers don't look at or for the same things.
2.3 Even if admissions essay content and style are correlated with income in ways that readers can notice or be unconsciously influenced by, that does not mean that these aspects of the essay have any effect of an admission decision.
2.3 Even if there is an effect, it's hard to predict which way it might go. A rich kid writing about vacations in Tahiti could be hurt by that essay. A poor kid writing about WIC might be helped.
2.4 In other words, if there is a correlation between income and essays, and if that correlation can affect admissions decisions, that still leaves us with a correlation that has little predictive power.
2.5 That's not true with tests.
2.5 That's not true with tests.
Problem 3. Tests and essays just ain't the same.
This is more about how the Stanford study has been used and interpreted by some people.
This is more about how the Stanford study has been used and interpreted by some people.
3.1 Some people look at the Stanford paper and say, "Hey, you think the SAT is an indicator of wealth? Well, look at the essay! It's even more correlated." This is a profoundly flawed, even silly argument.
3.2 The issue isn't really about the correlation between income and a component of an application but about the correlation between income and performance on a component likely to influence admissions decisions.
3.3 The Stanford paper did nothing to connect income, essays, and admit rates.
3.4 When it comes to tests, we know that, *broadly speaking*, the higher the income, the higher the test score, and the higher the test score, the higher the chance of admission.
3.4 When it comes to tests, we know that, *broadly speaking*, the higher the income, the higher the test score, and the higher the test score, the higher the chance of admission.
3.4 Show me (or an admissions officer) a student's essay and nothing else and they'll likely be no better prepared to evaluate a student's chance of getting in. Show me a test score and nothing else and I can make a prediction (esp for a low score).
3.5 Comparing the essay's correlation with income is nothing like comparing test scores' correlation with income. If you could show that income predicted weak essays that were more likely to get a student rejected, then you might have a leg to stand on.
3.6 The reason people push for test optional admissions is that the poorer you are, the lower your score is likely to be, and the lower your score, the less likely you are to be admitted.
3.7 It's either entirely disingenuous or deeply misinformed to equate the essay's connection to income with the tests'. Admissions officers aren't using essays & tests to identify a student's wealth. If they want that info, there are zip codes, high schools, and other items.
3.8 The connection between income & test scores is not iron clad, however, which is why I'm in favor of test optional admissions. There are low-income students who can really benefit from their scores and be seen. Students should have the choice to be seen & the chance to shine.
• • •
Missing some Tweet in this thread? You can try to
force a refresh
