Thread by @minethatdata

1 - In politics, some actively demean scientists.

In sports, you heard a prominent announcer last night say "sometimes you have to throw analytics in the dumpster".

Instead of focusing on the incidents/comments, focus on the bigger picture.

2 - I've told you before about being at a dinner with CEOs a few months ago, and one CEO did not like where my data would lead his company ... so he addresses me in front of the other CEOs and says the following:

3 - "Why should we listen to you? You're just a geek!"

4 - Now I put this flippin' wombat in his place with my response, given my conclusions are based on 250 companies and hundreds of billions of purchase transactions and his comment is based on general nastiness. But the comment is a symptom of a bigger trend.

5 - What is the bigger trend?

The bigger trend is a shift in power.

Over the past 10-15 years, people who accurately analyze information have acquired power ... maybe not as much as "data-driven" people would have expected, but far more than everybody else expected.

6 - This shift in power creates tremendous anxiety among those who traditionally possessed power.

7 - Let's use the Sunday Night Football example from last night, because that example represents the shift in power that exists.

The score was SF 24 / LA Rams 9.

The Rams score a touchdown, making the score 24-15. They now have a choice, with about three minutes left.

8 - If the Rams kick the extra point, they have a 95% chance of making it, making the score 24-16.

If that happens, they need to get the ball back, get a touchdown, then get a two-point conversion, and then they tie the score and go to overtime (in theory).

9 - However, if they get the touchdown (now trailing 24-22) and MISS the two point conversion, they're basically finished because there won't be much time left on the clock.

10 - Conversely, if they go for 2 now with about three minutes left, two things happen.

50% chance of getting it, leaving them down 24-17 and needing a touchdown and extra point to go to overtime.

50% chance of failure, down 24-15, needing two scores.

11 - The "analytics" show you clearly that going for two now protects you, because if you miss the conversion, you know already that you need two scores AND you have 3 minutes to accomplish the task.

If you fail going for two with 29 seconds left, well, no time left to adjust.

12 - If you follow the math on winning percentage, you learn that you have maybe a 10% chance of winning by kicking the extra point ... and maybe an 11% chance of winning (or greater) by going for two right now.

The numbers are bad either way.

But it is better to go for 2 now.

13 - So the analytics folks are recommending a strategy that goes against everything that long-term football people are used to. It runs counter to what feels "normal". And the expected gain is so small that it hardly seems worth having the argument in the first place.

14 - However, if you add up all of those 1% decisions over the course of a game, you have a 10% or 15% better chance of winning, and over the course of a season a 9-7 team becomes a 10-6 team or 11-5 team. So it is COMPLETELY worth following the analytics for an entire season.

15 - To follow the analytics, then, you should commit fully to it, and you have to take power away from a traditionalist and give it to somebody else.

It's the power shift that is the real problem if the reward isn't big enough to account for the shift in power.

16 - That's ultimately what we're dealing with.

If the President does what scientists want, the reward (to him) is too small (COVID still exists, restrictions are painful), and power is lost.

In sports, the experts lose their authority and there is no reward for them at all.

Analysts / Scientists aren't going to solve the "power" issue.

This is a generational issue, part of the pain introduced by fundamental change. It takes a generation of people trained in different thinking to replace traditional thinking. It's a long, hard, painful slog.

18 - And that's what we're going through right now ... the long, painful slog.

AND the data/analytical folks will always be incorrect at times, giving traditionalists something to point at and discredit / demean.

19 - You won't fix this problem by arguing loudly, or via a beautiful data viz.

You fix the problem over time by being more "right" than everybody else while holding your head high.

It's a long-term fight.

And you will likely win the long-term fight.

KH