Interested in mind wandering? In measurement (schmeasurement) and construct validation? After years in the making, I'm so proud--& relieved--that our new study preprint is available to check out. (THREAD)
psyarxiv.com/te9yc
psyarxiv.com/te9yc
2: Here's our abstract: 
3: Here's our ending recommendation & plea:
4: A central data figure showing that thought probes asking about MW content ("what"), MW intentionality ("why"), & MW depth ("how much") all yield similar TUT rates in our 1st task/SART, but not in the 2nd/flanker:
5: One striking difference among probe types was that content-based probes yielded higher confidence in MW reports than did intentionality or depth probes--and more similar confidence to on-task reports:
6: All probe types showed a bit more RT variability preceding MW reports vs on-task reports. BUT intentionality & depth probes elicited MUCH bigger accy diffs b/w MW & on-task reports than did content probes. We argue that intent & depth MW reports = more reactive to performance.
7: Correlation table: Content probe MW rates more consistently correlated w/executive-task performance; Intentionality probe MW rates more consistently correlated w/retrospective Qs about trait distractibility & daydreaming; maybe more evidence 4 intentionality probes reactivity?
8: Lots of recent studies probe thoughts with 1-5 Likert scales (anchored by completely on-task--completely off-task). Researchers have assumed a quantitative continuum b/w on/off-task thought w/o serious theorizing or testing the assump:
9: We wondered whether Ss confounded their 1-5 MW depth rating with their confidence in the report: Would they use the middle point to indicate being unsure, rather than an intermediate state b/w on- and off-task. Here's confidence by MW rating in the SART task:
10: Weirdly, confidence goes down as Ss report deeper MW. But note in that figure above some bimodality in confidence for the deepest MW rating (5). Lots of Ss highly confidence & lots very unconfident. Next tweet looks at them separately.
11: Here's the 1/3 of sample w/highest confidence in deepest MW rating--they show shallow U-function suggesting they used mid-point to indicate intermediate confidence, at least to some degree, and perhaps NOT to indicate intermediate conscious experience.
12: Here's the 1/3 of sample w/lowest confidence in deepest MW rating--whoa. They lose confidence dramatically as they indicate deeper MW. Either they're also confounding depth & confidence (differently than above), or they have very poor meta-awareness.
13: The fact that diff Ss seem to be using the 1-5 MW depth scale so differently calls the validity of these reports into serious question. We don't know what they're actually measuring, and we recommend not using them w/o more validation work.
14: We think there are lots of other cool findings & arguments in the paper (e.g., concerning how to validate Ss' reports of intentional vs. unintentional MW), & we're proud of our methodological approach, so we hope you'll check it out.
15: A heads-up to potentially interested parties who may have not seen this thread yet: @JkayFlake @EikoFried @the_mindwanders @Paul_Seli @JonathanSchool6 @doctorwhy @matthewkrobison @KalinaChristoff @sbkaufman @jan_rummel @psycdelius
