Thread by @AlamedaTrabucco

Look at these two images -- one is my comment on Slack at 4:05am (Asia time), and one is BTC's price graph at 4:06am. Am I the best trader in the world?

A thread about decisions and results.

Many disciplines revolve around decision making. Often, the decisions are pretty easy -- especially for someone who's experienced in the discipline. The difference between top performers and everyone else tends to be how they do on average at the few % of the hardest decisions.

A key feature of those decisions is that most people -- non-experts in their fields -- would sometimes get them wrong, or at least not have a great framework for evaluating them. And that can lead to over-reliance on results when evaluating the decision itself.

Let's look at a dumb example. Say you've got a die, and I bet you 1:1 the next roll will come up 6. You agree, and the die comes up 6. I won! But did I make a good decision? Most would say (correctly) that I did not.

(Unless I swapped a loaded die in when you weren't looking :P)

But when the decision is one that most people have more trouble evaluating than "what's the probability of a die roll coming up 6," people DO assume that results carry too much information.

Let's look at some examples from "real life" (I'll get back to trading in a bit):

The TV show "Survivor" features two teams competing in mostly physical competitions, with the losing team voting someone out. At ca. 10 people left the teams combine, and the vote-outs continue til a few people remain. The jury (last few eliminated) then vote for a winner.

On season 20, a guy named J.T. had a "hidden immunity idol," which grants safety for one vote. J.T.'s team had 5 people, the other team had 6 -- notably, the other team was down to 5 women and 1 man. That's SUPER weird for "Survivor" since physical strength matters a lot early.

J.T. had the (IMO) genius idea to give his idol to Russell, the last man on the other team -- he figured the women had teamed up (which one woman on the team, Parvati, had done before), so Russell was next. Then at 10 left, they'd merge and J.T.'s team + Russell would be up 6-4.

Alas -- Russell was working with the women (he was sort of in charge). At 10, he used J.T.'s idol to eliminate J.T., and his team followed.

Many call J.T.'s move the WORST ever made -- but that's completely clouded by the results, and the move would usually have worked great.

Look at this list of the worst coaching decisions ever made in the NFL: sportscroll.com/30-nfl-coachin…

In particular, look at #2 on the list -- it's Pete Carroll's (in)famous call during Superbowl XLIX.

The Seahawks were down by 4 with 26 seconds left at the Patriots' 1 yard line, 2nd down. The more standard decision here would have been to attempt a running play -> Marshawn Lynch, who was notably dependable in spots like that.

But Carroll called for a pass.

The pass got intercepted and the Seahawks lost the game -- fans immediately started calling the decision crazy, Pete Carroll the worst coach in NFL history.

But ... it's obviously not that simple.

It was 2nd down, and ideally they'd run the clock down as much as possible before scoring -- the Patriots had some chance to come back if they didn't. 3 running plays wouldn't fit into 26 seconds -- but a pass and 2 running plays (or 2 passes + a running play) could.

I'm glossing over some details, but the REAL decision here was basically a trade-off between giving the Patriots time to score (a 5% or so event pending how long it was) vs. the % of an interception (1-2% or so per attempt) ... doesn't look so bad after all.

(Some more detailed analysis which doesn't gloss over the details that I did can be found here: fivethirtyeight.com/features/a-hea…)

You see people assess decisions all the time which are based more on results than anything. And that's natural! When evaluating a tough decision, often the results are the only "hard info" many people have in practice (since math about interception % isn't practical).

Every time someone has folded a hand and asked to see the river "so they know," or sworn off of drinking tequila because of one bad night when they drank too much -- they've acted in a results-oriented way instead of a statistically sound one.

When trading, I try to totally divorce myself from results. If someone asks for feedback on a decision, I try to not even ask how it went, and I work really hard not to over-update my own future behavior based on one trade of mine going especially well or badly vs. expectation.

Note that this is hard to balance vs. the times when it IS important to update -- this is where Bayesian statistics come in. In general, I'm talking about single decisions where one result mostly isn't enough to update priors strongly -- different ballgame if priors are weak.

But one correct delta call -- or even, like, 4 correct delta calls -- isn't enough to label someone a strong trader. People on Twitter are very quick to call out traders who made well-timed predictions -- but if you're not tracking EVERY decision they've made, you don't know ...

... if they just got lucky on a coin flip, or if they've been 60-40 on all the coin flips they've EVER called because they've got some edge.

(Note: I am often personally given WAY too much credit for calls I've made :P)

So I basically never update about someone's skill because they did one thing right -- or about my own! It's the same reason crypto exchange leaderboards don't tell you much, and people who just post delta calls are rarely worth listening to without running lots of analysis.

I recommend thinking hard about whether you're updating too hard in response to your own wins and losses -- getting better at this was a huge key for me to improving at trading, which is about making thousands of good calls, not just one.

To be clear, the answer was "no" to my initial question, also in this case we didn't really buy anything before the spike because I thought of it too late :(