Thread by @JonBoeckenstedt

Thread: By now, if you work in higher ed, you've almost certainly seen this NY Times article by @paultough Even though I'm in it, I think it's really good. nytimes.com/interactive/20…

It's an excerpt/companion piece to his new book, The Years That Matter Most: How College Makes or Breaks Us amazon.com/Years-That-Mat…

From what I've seen in the book, and from reading the excerpt, it is as good a treatment of what people in Enrollment Management deal with on a daily basis as I've seen. Paul was very thorough, and a quick learner, and very patient as I tried to explain all this stuff.

(I've often said there needs to be a German word for that situation where you can't explain B until the person understands A, and you can't explain A until the person understands B. That's the great challenge in putting our work into some context. Everything is related.)

Anyway, some of the book is not complementary to The College Board. And the College Board responded, saying, essentially, "We let you in, and gave you access, and you repay us by telling the truth? THE NERVE!" Here is the response pages.collegeboard.org/paul-tough

There is another article about the CB response here, which must have been a while in the making, or it was done by the fastest writers in the country, which makes me think it was pushed as a story by CB. Just a guess though chalkbeat.org/posts/us/2019/…

What you see by comparing these two pieces (the CB piece actually sites the Chalkbeat piece) is the College Board talking tough on its own internal memo, and then backing off in comments that tend to stick with you later on.

For instance, Tough says the packets project was unsuccessful (stuff happens, and that's not a problem) but CB should have reported on it. CB says WE DID (but not until the book went to press, duh). Then, Coleman says this:

The interesting thing, it seems, is that College Board will "own" stuff only after people call them on it. It's this exact arrogance and hubris that turns people away from the organization in the first place. And yet they never learn.

The same thing surfaces with Khan. Anyone who listened to the 20 hours/115 points knew it was almost certainly bullshit. It wasn't a study where students were randomly selected to do Khan or not do Khan, and then compare outcomes. Puffery? Not a big deal?

These people are statisticians. And their statistics are what determine to a large part who goes to college where in this country. And they can't even do that right, it seems, based on the wild swings in raw-to-scaled conversions since they took test creation back from ETS.

And, of course, the Alanis Morissette moment here is this in the Chalkbeat piece. Do you see it? Research shows test prep doesn't work, so we designed Khan Academy test prep which does.

And this, in which Coleman is apparently tired of defending the indefensible:

Everyone knows the SAT and ACT add a teeny tiny bit to the prediction equation. So literally, more information is better. But two points: It's obvious the reason it adds little is because it largely echoes HS GPA. So for most students it's counting the same thing twice.

The question is--once again--is this 2% of variance in freshman grades explained by adding SAT worth it? When it keeps thousands of kids out of college? When it's misused to compare high schools and districts? When it's used to allocate tax dollars?

When it takes as much as 15 percent of classroom time away from instruction? When journalists misinterpret the results? When CB makes up "College Ready" stats? When--on those occasions when it does give someone a boost--that person is almost always a white or Asian male?

What about when it's used in statistically inappropriate ways, like at Harvard (revealed in last year's trial?) Or when colleges tell students: We like a 780 a lot more than a 760? Is it worth it?

Are they listening to these objections?

Or do they--with a billion or so dollars in assets--still insist that there is "no mercantile motive?" Yes, David Coleman said that.

It's this arrogance and defensiveness when they are called out on their blunders (the Parkland email; the non-existant test security; Landscape)

That causes people to not believe a single thing the College Board says.

Why don't they get this?

People think I hate the College Board and ACT. I don't. I've said before I like most of the rank-and-file people there, and I know they believe their working there for good reasons.

It's just a private company. Use their products or don't.

But I do resent their self-appointed intrusion into the American primary and secondary education system, and their stranglehold as major sources of inequity in American higher education.

And of course, the emotional, heart-tugging justification for all this is the "Diamond in the Rough" concept they love to push. If you're that kid who wins the lottery (an apt metaphor) and scores highly despite your circumstances, it's all worth it to them.

And don't get me started on the "grade inflation" canard they like to drag out. School assessment tests and norm-referenced tests are different, and--surprise--scores on the latter are also going up (although ACT makes it easier to prove this than College Board does)

The gist: College Board is acting exactly the way you would predict a threatened business to act. You can't blame them for that.

But you can come to your own conclusions on whether their stance makes sense against their long history of arrogance and duplicity.

Oh, and #TalkingAboutEM