Crémieux Profile picture
Aug 24, 2024 19 tweets 6 min read Read on X
What do the Washington Post, Brookings, The Atlantic, and Business Insider have in common?

They all employ credulous writers who don't read about the things they write about.

The issue? Attacks on laptop-based notetaking🧵


Image
Image
Image
Image
Each of these outlets (among many others, unfortunately) reported on a a 2014 study by Mueller and Oppenheimer, in which it was reported that laptop-based note-taking was inferior to longhand note-taking for remembering content. Image
The evidence for this should not have been considered convincing.

In the first study, a sample of 67 students was randomized to watch and take notes on different TED talks and then they were assessed on factual or open-ended questions. The result? Worse open-ended performance: Image
The laptop-based note-takers didn't do worse when it came to factual content, but they did so worse when it came to the open-ended questions.

The degree to which they did worse should have been the first red flag: d = 0.34, p = 0.046.
The other red flag should have been that there was no significant interaction between the mean difference and the factual and conceptual condition (p ≈ 0.25). Strangely, that went unnoted, but I will return to it.
The authors sought to explain why there wasn't a difference in factual knowledge about the TED talks while there was one in ability to describe stuff about it/to provide open-ended, more subjective answers.

Simple: Laptops encouraged verbatim, not creative note-taking. Image
Before going on to study 2: Do note that all of these bars lack 95% CIs. They show standard errors, so approximately double them in your head if you're trying to figure out which differences are significant.

OK, so the second study added an intervention.
The intervention asked people using laptops to try to not take notes verbatim. This intervention totally failed with a stunningly high p-value as a result:Image
In terms of performance, there was once again nothing to see for factual recall. But, the authors decided to interpret a significant difference between the laptop-nonintervention participants and longhand participants in the open-ended questions as being meaningful. Image
But it wasn't, and the authors should have known it! Throughout this paper, they repeatedly bring up interaction tests, and they know that the interaction by the intervention did nothing, so they shouldn't have taken it. They should have affirmed no significant difference!
The fact that the authors knew to test for interactions and didn't was put on brilliant display in study 3, where they did a different intervention in which people were asked to study or not study their notes before testing at a follow-up.

Visual results: Image
This section is like someone took a shotgun to the paper and the buckshot was p-values in the dubious, marginal range, like a main effect with a p-value of 0.047, a study interaction of p = 0.021, and so on

It's just a mess and there's no way this should be believed. Too hacked!
And yet, this got plenty of reporting.

So the idea is out there, it's widely reported on. Lots of people start saying you should take notes by hand, not with a laptop.

But the replications start rolling in and it turns out something is wrong.
In a replication of Mueller and Oppenheimer's first study with a sample that was about twice as large, Urry et al. failed to replicate the key performance-related results.

Verbatim note copying and longer notes with laptops? Both confirmed. The rest? No. Image
So then Urry et al. did a meta-analysis. This was very interesting, because apparently they found that Mueller and Oppenheimer had used incorrect CIs and their results were actually nonsignificant for both types of performance.

Oh and the rest of the lit was too: Image
Meta-analytically, using a laptop definitely led to higher word counts in notes and more verbatim note-taking, but the performance results just weren't there. Image
The closest thing we get in the meta-analysis to performance going up is that maybe conceptual performance went up a tiny bit (nonsignificant, to be clear), but who even knows if that assessment's fair

That's important, since essays and open-ended questions are frequently biased
So, ditch the laptop to take notes by hand?

I wouldn't say to do that just yet.

But definitely ditch the journalists who don't tell you how dubious the studies they're reporting on actually are.
Sources:





Postscript: A study with missing condition Ns, improperly-charted SEs, and the result that laptop notes are worse only for laptop-based test-taking but not taking tests by hand. Probably nothing: journals.sagepub.com/doi/10.1177/09…
journals.sagepub.com/doi/full/10.11…
journals.sagepub.com/doi/10.1177/00…

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Crémieux

Crémieux Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @cremieuxrecueil

Oct 22
The story of peanut allergy is entering its final chapter.

Nowadays, we are beating back both peanut and other food allergies, and all it took was telling parents the right thing to do🧵 Image
The story begins in 2000, when the American Academy of Pediatrics decided to issue some simple advice to parents: Have your kids avoid peanuts early in life. Don't expose them until they're at least three!Image
Parents complied. It turns out, they do that. They just follow advice from professional associations that appear to have authority.

So peanut allergy rates rose, from 0.4% in 1997 to 1.4% in 2008, to 2% in 2015.

But if the advice was right, the opposite should've happened!
Read 13 tweets
Oct 21
The CEOs managing Sweden's biggest companies tend to be smarter, taller, and to have better personalities. Image
The highest-paid CEOs in Sweden also tend to be taller, smarter, and to have better personalities. Image
To connect these things, CEO pay and firm size are considerably connected. Image
Read 4 tweets
Oct 21
Individuals with higher genetically predicted intelligence tend to end up working more prestigious jobs. Image
They tend to be more emotionally stable, healthier, satisfied with life, richer, and educated too. Image
Additionally, they tend to be happier and to have fewer psychiatric problems.

They're more likely to be go-getters! Image
Read 4 tweets
Oct 19
Boobtech is amazing.

It's an area that the rest of medicine could look to as an example.

The professionals making bigger, more realistic breast implants are simultaneously improving affordability, safety, and quality at a rapid rate🧵 Image
Consider one of the most recent improvements in boobtech: the Mia.

The Mia is the first successful "injectable" breast implant.

It cuts down scarring, complications, surgery time and cost, and it looks and feels more realistic than earlier implants. Image
The Mia is installed with a small armpit incision about 2 centimeters in length.

This is a significant reduction from earlier generations, which were regularly closer to 7 centimeters, or almost 3 inches. Image
Read 15 tweets
Oct 18
This should be considered *far* more alarming than the polls about political violence.

Two-thirds positive views towards an evil ideology that has killed tens of millions and cannot work is *very* bad. Image
It doesn't really matter if, at the end of the day, they're actually tepid towards socialism. This is like 66% of people saying Hitler was OK.

Source: news.gallup.com/poll/694835/im…

And an article qualifying how we understand support for political violence: cremieux.xyz/p/lets-not-ove…
I get too many dumb comments.

The dumb comment for this post is going to be something along the lines of 'But they're thinking about [successful place] not [bad place]!'

Ten points if you realize why that is not a meaningful reply.
Read 9 tweets
Oct 17
This is not true and there has never been a reason to believe it.

When we do have raw data for anywhere, we see that there's consistent scoring over time, not massive intelligence gains.

If we do not take measurement invariance seriously, we will be seriously misled. Image
I actually think it is exactly Noah's sort of post that helps to keep the culture of scientific fraud in academia and elsewhere alive.

Noah is smart enough and has been told enough to know better, and he still wrote something that he can't support.

But it's a popular message.
The message is just empirically wrong.

Will we ever move beyond the Cargo Cult version of the Flynn Effect that people like Noah, knowingly or otherwise, are wont to promote?

I don't think we will!

To learn more, see:
Read 5 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(