Thread by @SpencrGreenberg

Does astrology work? We tested the ability of 152 astrologers to see if they could demonstrate genuine astrological skill.

Here is how the study was designed and what we found (including a result that really surprised me):

🧵

Back in January, we ran a study trying to predict 37 facts about people's lives using their astrological sun signs (whether they are Pisces, Aries, etc.) While personality tests were able to predict these facts decently well, sun signs couldn't predict even a single 1 of them...

Some astrologers criticized us for this, saying that sun signs/zodiac signs are just tabloid astrology - real astrologers use a person's entire astrological chart.

And they're right!

Taking into account this criticism, we got the help of 6 astrologers to design a new study.

Here's how the study worked to test astrologers:
• in each round, each astrologer gets LOTS of information about a real person (answers to 43 questions) along with 5 full astrological charts
• they then predict which is the person's real natal chart (the other 4 are decoys)

Why this study design?

One of the most fundamental claims of astrology is that a person's natal chart contains information about that person's life and character.

If true, astrologers should be able to correctly choose a person's chart at a rate well above random guessing.

Each astrologer tries to match people to their correct chart 12 times. If they're guessing completely at random (e.g., they have no skill because astrology doesn't actually work), then they'll get about 20% of questions right, or about 2.4 questions right (on average) out of 12.

Neat aspects of this study design are that (1) if astrology doesn't work, it's impossible for astrologers to do better than random guessing at this task, while (2) for the study to come out in support of astrology, astrologers only need to do slightly better than random guessing

But this is only a fair test if astrologers believe they can do this task - so we limit our analyses only to participants with prior astrological experience who predicted they would do better than random guessing at the task. Our results are based on 152 such astrologers.

These astrologers were quite confident in their ability to match people to charts. Those with the least experience believed (after they had completed participation) that they'd gotten 5 out of 12 right, and those with the most experience thought they'd gotten 10 out of 12 right.

So, how did astrologers do overall? If they'd gotten even 23% of questions right (slightly above the 20% of random guessing), the study would have come out in favor of astrology. But astrologers as a group performed indistinguishable from random guessing, getting < 21% right.

We can compare how frequently astrologers got different numbers of questions correct to how often we'd expect them to get different numbers correct if they were all guessing totally at random with no skill.

The two distributions match very closely.

But perhaps the less experienced astrologers were just dragging down the performance of the group?

We looked at how performance varied based on astrological experience. More experienced astrologers did not do better than less experienced ones despite being far more confident.

Even if most astrologers have no skill, there's another way astrology could prove itself. If even 1 of the 152 astrologers performed exceptionally well, that could provide meaningful evidence for astrology. We offered a $1000 prize for anyone getting at least 11 out of 12.

Unfortunately, despite more than half of the astrologers believing that they had gotten 6 or more questions right (after completing the task), in actual fact, not a single astrologer got more than 5 right.

Okay, so despite them believing they could do this task, astrologers seemed to have no ability to match people to their astrological charts. But, even if they aren't getting the answers right, do they at least agree with each other on what the right answers are?

Much to my surprise, astrologers had very low agreement with each other on the chart for each person. If astrologers picked charts at random, they would agree with each other 20% of the time. In our study, even the most experienced astrologers only agreed 28% of the time.

In conclusion, despite believing they could do it, the 152 astrologers seemed to lack any ability to match people to their astrological charts.

You can learn a lot more about the study (including its limitations and how we sought to address them) here:

clearerthinking.org/post/can-astro…

If you believe you have astrological skill, you can try the same questions that we used in the study (and find out the right answers at the end) in order to test yourself:



We also open-sourced the data from the study if you want to analyze it.programs.clearerthinking.org/astrology_chal…

If you found this interesting, I'd appreciate a follow at @SpencrGreenberg !

You may also enjoy my newsletter (One Helpful Idea) - where I send out one idea weekly (a 30 sec read) about psychology, philosophy, or society:

bit.ly/onehelpfulidea

If you enjoyed our astrology work, you might also like our "ultimate personality test" - which analyzes your personality using the 3 most popular personality frameworks simultaneously while teaching you about how accurate (or not) those frame,works are:

programs.clearerthinking.org/personality.ht…