Profile picture
Paras Chopra @paraschopra
, 8 tweets, 4 min read Read on Twitter
1/ My friend @arunbansal forwarded me this article.

Regarding probabilities, it is easy to get confused unless you communicate clearly what it represents.
@arunbansal 2/ There are two types of uncertainties associated with the small world and the big world.

Your model is the small world and the big world is, well, the true reality.
@arunbansal 3/ Small world uncertainty is how uncertain is your model given all the info available.

Big world is how uncertain YOU are about your model. It expresses your confidence of how closely the model tracks the real world.
@arunbansal 4/ Your model could be absolutely certain about predictions yet model could be completely wrong about real world.

Model is a model. It’ll do what you ask it to do. It wouldn’t know about assumptions and details that you don’t bake into it.
@arunbansal 5/ So it pays to clarify what type of uncertainty you are talking about.
@arunbansal 6/ Also, in reality, accuracy is a useless metric.

When you do an A/B test, you should optimize for profit and not for accuracy. That is how @VWO’s statistical engine also works. It gives you a smart decision first and then asks you to shoot for accuracy.
@arunbansal @VWO 7/ What you need us a cost function that maps predictions and probabilities to decisions about what you’ll do in the real world.

If acting on even a 10% probable event can give you profit, why would you wait for 90% probability?
@arunbansal @VWO 8/ Ultimately, profit and survival trumps accuracy.

You don’t get paid (by customers of employers) to predict accurately, you always get paid to take good decisions in a real world context.
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Paras Chopra
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member and get exclusive features!

Premium member ($3.00/month or $30.00/year)

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!