,
11 tweets,
4 min read
Read on Twitter
“Tests For the 'Big Five' Personality Traits Don't Hold Up In Much of the World”
This article is a fantastic example of misleading and inaccurate science reporting. [THREAD] 1/
blogs.discovermagazine.com/d-brief/2019/0…
This article is a fantastic example of misleading and inaccurate science reporting. [THREAD] 1/
blogs.discovermagazine.com/d-brief/2019/0…
Is the Five Factor Model (FFM) WEIRD? Are the Big Five personality traits — statistically, the factor structure yeilding 5 traits — only valid in western countries?
Well, no. The research article shows primarily a methodological problem, not a cultural problem, with the FFM. 2/
Well, no. The research article shows primarily a methodological problem, not a cultural problem, with the FFM. 2/
Problem 1. The article has three data groups. A US dataset, an cross-cultural internet set, and a cross-cultural face-to-face interview “survey” dataset.
Important: both cross-cultural sets have data from the SAME countries. 2/
Important: both cross-cultural sets have data from the SAME countries. 2/
Also, total N ~300K across all three datasets. The 90K N is for only the face-to-face survey dataset.
Note. The article refers to the face-to-face dataset as the “survey data” or STEP data throughout. 3/
Note. The article refers to the face-to-face dataset as the “survey data” or STEP data throughout. 3/
So, is the FFM of personality valid only in WEIRD countries?
No. The FFM is not valid in the survey/STEP data. It is valid in the internet sample in the SAME countries.
Thus “cultural diffs are unlikely to be the main driver of low validity in the survey data” 4/
No. The FFM is not valid in the survey/STEP data. It is valid in the internet sample in the SAME countries.
Thus “cultural diffs are unlikely to be the main driver of low validity in the survey data” 4/
Statistically, the correlations between items within each factor (“trait”) reproduce pretty well in the US and cross-cultural internet sample.
Again, the results differ for the cross-cultural survey dataset.
This is suggestive of a methodological, rather than cultural prob 5/
Again, the results differ for the cross-cultural survey dataset.
This is suggestive of a methodological, rather than cultural prob 5/
The article investigates numerous methodological reasons, as well as WITHIN country/region differences for the diffs btw internet and survey data in the SAME countries. For example, participants taking test online vs in person differ.
Not one conclusive reason, but several 6/
Not one conclusive reason, but several 6/
the main conclusion of the article findings — that the lack of FFM structure in the survey data vs the internet data in the SAME country — is NOT primarily a result of cultural differences.
The science report presents a conclution directly opposed to the articles main result 7/
The science report presents a conclution directly opposed to the articles main result 7/
This inaccurate reporting is unfortunate. Broad conclusions based on only WEIRD data are very real problems in behavioral sciences.
Misrepresenting data to fit a narrative only serves to increase skepticism of real issues & increases public mistrust of scientists. /end
Misrepresenting data to fit a narrative only serves to increase skepticism of real issues & increases public mistrust of scientists. /end
And, here is another write up of this research from @medical_xpress that also fails to accurately report what the research actually found, simply avoiding discussion of the other 190K participants in the study
