Thread by @_psyguy

Here's my internship paper with Francis Tuerlinckx on #measurement/#psychometrics of affect & #ESM using graded-response #IRT.
The main finding:
The state & trait latent constructs of affect—contrary to the common belief—are essentially different.
Summary in 9 tweets:
#THREAD 1/9

https://twitter.com/PsyArXivBot/status/1167867813060431872

Nonergodicity has been heavily ignored in psychology. In the intro we give an extensive historical overview of how this has troubled our idiographic (person-centered) and nomothetic (~individual differences) understanding in psychology—and the way people have tried to fix it.
2/9

In #ESM, sum-scores of PANAS-like positive & negative items are used as constructs of affect—but is that a valid decission?
To account for non-normality (& to allow nonlinearity & check latent nonmonotonicity), we used graded-response item response theory for factor analysis.
3/9

We studied 12 datasets of ESM/daily diaries (studied recently by @DejonckheerEgon and @merijn_mestdagh et al.) and made two pools of samples for each wherein the within- and between-person (WP & BP) variations were disentangled (~slicing Cattell's Data Cube, second fig here).
4/9

The samples were modeled by 9 possible factor structures (Valence, PA-NA, PA-NA-3rdFactor, Valence-Arousal, Valence-Arousal-3rdFactor; with & without latent covariances), & compared based on their AICc (it's important why this information criterion, not others; cf. endnote 4)
5/9

For reasons mentioned in the text, multi-group/multi-level IRT modeling was not viable, hence we relied on vote-counting:
For each pool of samples, we counted the number of times each factor model has had the lowest AICc.
6/9

Overall, the best-fitting factor structure underlying intraindividual variations of affect is unidimensional, i.e., Valence. Conversely, interindividual differences are better explained by the 2D PA-NA construct.
This is consistent (cf. the Discussion) across all 12 datasets.
7/9

The PANAS-like instruments of #ESM measure BP decently (cf. factor loadings and h2 in first plot). However, it sucks at measuring WP variations (cf. second plot).
The latter point has a very important implication: These instruments are not suitable for "experience" sampling.
8/9

The paper concludes with an illustrative example of how an instrument can measure (incongruent) state-like and trait-like constructs at the same time.
Also, find the presentation of these findings (w/ notes on monotonicity) at @BapSciences 2019 on OSF:
osf.io/rn8c6/
9/9