Thread by @SarangSood

SURVIVORSHIP BIAS

Survivorship bias is the tendency to analyze the strategies that performed exceptionally well and ignore all those that didn’t reach our criteria. We have an inclination towards focusing on the survivors, even if they survived out of luck rather than viability.

So basically we are testing our ideas on an incomplete data. The missing data may conceal losing trades or winning trades, causing us to deploy a strategy which we shouldn't or vice versa. Backtest results with this bias will not represent the true picture.

So if we have a backtested strategy with SL on a 5 min time frame (or even 1 min for that matter), it may give a totally different outlook to the actual result. The period in between which is not taken into account can give vicious price movement hitting the SL multiple times.

Many people take advantage of survivorship bias by using simple probabilities to convince retailers that their strategy is performing exceedingly well, by backtesting on incomplete data. Retailers blame their result on trading psychology, but the truth is something different.

It is one of the most common biases in finance & one which is easy to fall victim to. If you understand why survivorship bias occurs and you recognise the effect, it will help you better understand your data and make your analyses more reliable and valid.

Famous War Plane example to give more perspective:

During World War II, Abraham Wald and his colleagues were commissioned by the government to study long-range bombers, a high risk job for pilots back then. Their sole task was to figure out how to keep more bombers in the sky.

The research started with simple observation. After a bomber returned from enemy territory, it was examined for damage. Data were graphed for each surviving plane. After a while a pattern emerged. The common damage occurred in the tail area, the middle underbody, and the wings.

Wald’s colleagues concluded that these were the most vulnerable parts of the plane and should be reinforced with extra armor. The front and rear undersides of the planes hosted little or no damage, so they shouldn't be modified.

But Wald a mathematician by trade theorized the opposite.

Wald argued that the parts where damage occurred the most showed strength & not weakness, revealing exactly where the bombers can be hit and still survive.

He added further that the most vulnerable parts of the plane were those which weren't damaged. These were the parts which needed reinforcements because planes exhibiting damages to those parts were blown away, not surviving the flight home.