Thread by @tervoclemmensb

Now at @nature ! Work I co-first authored w/@smarek0502 :

Brain-wide association studies require thousands of individuals

nature.com/articles/s4158…

The result of years of graduate and post-doc work, #teamscience, and learning to embrace scientific complexity.

1/x

Note this work is not about all neuroimaging, but very specifically about cross-sectional correlations between individual differences in functional/structural neuroimaging metrics and psychological phenotypes. Please read and share with appropriate nuance.

With data from N ~50,000 (ABCD, HCP,UKB), we tested the magnitude and reproducibility of associations between common, inter-individual variability in human brain structure/function and cognition and psychiatric symptomatology: Brain-Wide Association Studies (BWAS).

In ABCD, we found that the largest replicable univariate BWAS associations were |r|=.16, top 1% |r|=.06. What do these univariate effects mean for smaller BWAS linking inter-individual variability in brain structure/function to psychological phenotypes?

In simulated smaller studies, univariate effects, coupled with sampling variability, assured statistical errors in BWAS among sample sizes typical for neuroimaging (N<500). Univariate BWAS improved, but replication remained low, as samples reached the thousands.

The magnitude of BWAS effects was consistent across ABCD, the single site adult human connectome project (HCP), and the large UKB dataset with subtle, per-sample differences well accounted for by sample size differences.

Adjusting for sociodemographic covariates further diminished BWAS effects.

Multivariate methods improved reproducibility of BWAS in our simulations, but likewise were
constrained by sample size.

!”There is no one-size-fits-all for neuroimaging studies; minimum sample size requirements depend on the study aims. Neuroimaging-only studies are typically adequately powered at small sample sizes.”! (see paper).

!Non-BWAS: principles of brain organization, method dev., and context specific activation can all be reproducibly studied with small samples! We are specifically suggesting to recontextualize how we approach BWAS: brain structure/function <-> individual differences in phenotypes.

We argue to adapt and expand non-BWAS designs that link brain and behavior through longitudinal/within-person and/or intervention studies, rather than observational and cross-sectional BWAS. Further methodological advances in imaging+behavior will likewise benefit all studies.

BWAS should use the largest sample sizes, replicate out-of-sample, and continue to acknowledge methodological and interpretive complexity. Following the example from GWAS, large neuroimaging consortia and data aggregation efforts are accelerating these BWAS efforts.

Huge thank you to co-senior authors @ndosenbach and @DrDamienFair who provided unyielding support to me and @smarek0502 throughout this process.

ICYMI this work is started with initial observations made in our DCN paper.

sciencedirect.com/science/articl…

P.S. Neuroimaging is essential to our understanding of brain function/structure. Our work is specific to BWAS and we do not agree with any sweeping statements regarding neuroimaging writ large based on our results. Please read and share this paper with appropriate nuance.

P.P.S. We define BWAS formally in the paper and encourage further dialogue on distinctions between different types of brain-behavior linkages. Again, there is no one-size-fits-all for neuroimaging studies.

PS. X3. For additional, quant methods focused (non neuro; all simulation) perspective on stabilization of correlation, sample size, effect size, and other params. be sure to check out seminal work from Schönbrod & Perugini if you aren't familiar. sciencedirect.com/science/articl…

*Schönbrodt & Perugini

Ps x4 in addition to our direct analogy to GWAS, there are examples, theory, and debates to be drawn from in other areas with challenges and success in multi method agreement (cf., cognitive models versus symptom reports of impulsivity).