,
5 tweets,
4 min read
Read on Twitter
Last year, @NIMH_RDoC @NIMH commissioned a report from us @McLeanHospital (w/ @DanielGDillon @JustinBakerMD @laurenrutterr) re digital cognitive assessments for field test use, from >300k TestMyBrain participants. That report is now available! psyarxiv.com/dcszr/
Key findings: First, (1) *RELIABILITY IS A MAJOR ISSUE*, particularly for tasks from basic science / experimental traditions, built for detecting within person diffs and where reliability for between person is rarely reported. #psychometrics <- probably not trending
Second, (2) measures of negative valence / emotion (e.g. threat processing) and positive valence (e.g. reward sensitivity) usually not built for large-scale mobile assessment: technically, psychometrically, or re engagement - important for projects like @AllofUsResearch
Third, (3) Device variability is an issue for #digitaltechnology, particularly mobile cognitive assessments. Devices differ! Measurements of response time latency can differ by as much as 100ms between different smartphones. See also tandfonline.com/doi/full/10.10…
Fourth, (4) and can't stress this enough: participant engagement matters. We have N=300k in #citizenscience because the participant is a key stakeholder. And (surprise! win-win!) tests with better psychometric properties had better participant ratings and better retention!
