,
16 tweets,
3 min read
Read on Twitter
Here's a fun setup: selection filters as stochastic decision processes, super simplified.
1. You pick a "value ball" from an urn
2. Each ball is two triples of {profit, probability, arrival time}, one with treatment, the other without
3. You can buy one treatment outright
1. You pick a "value ball" from an urn
2. Each ball is two triples of {profit, probability, arrival time}, one with treatment, the other without
3. You can buy one treatment outright
4. Treatment changes at least one variable, but isn't observable
5. You can also buy unlimited tests
6. Tests updates your info about at least one variable in your current treatment
7. Variable cost, "burn rate" is constant until arrival time
8. You can terminate at any time
5. You can also buy unlimited tests
6. Tests updates your info about at least one variable in your current treatment
7. Variable cost, "burn rate" is constant until arrival time
8. You can terminate at any time
9. Total burn is negative payoff if you terminate before arrival time
10. Payoff at arrival is profit based on treatment minus burn
11. Add discounting and other embellishments if you feel like it
10. Payoff at arrival is profit based on treatment minus burn
11. Add discounting and other embellishments if you feel like it
Now add some fun complications about treatments and tests:
12. Of all observed paths to arrival, treatment has produced higher net profit than no profit
13. That doesn't mean any individual treatment will
14. E.g. treatment can increase profit but lengthen path to arrival
12. Of all observed paths to arrival, treatment has produced higher net profit than no profit
13. That doesn't mean any individual treatment will
14. E.g. treatment can increase profit but lengthen path to arrival
15. There is no way to tell if a test gets you closer to the true value of any variable
16. Tests can be purchased from experts
17. Experts have an-above average profit portfolio for all paths on which they administered tests
18. You can't observe *any* counterfactual paths
16. Tests can be purchased from experts
17. Experts have an-above average profit portfolio for all paths on which they administered tests
18. You can't observe *any* counterfactual paths
And a few more:
19. Tests are both cost and effort intensive, so will move back arrival time
20. You can use tests and experts at any time, concurrently or sequentially, but
21. Experts can and will disagree over variables, leaving you to interpret their estimates
19. Tests are both cost and effort intensive, so will move back arrival time
20. You can use tests and experts at any time, concurrently or sequentially, but
21. Experts can and will disagree over variables, leaving you to interpret their estimates
And if that's not enough:
22. Treatment can't just be applied at the beginning, but anywhere along the path at any intensity, leading to an unobserved triple {profit(τ), probability(τ), arrival (τ)} depending on treatment intensity τ
22. Treatment can't just be applied at the beginning, but anywhere along the path at any intensity, leading to an unobserved triple {profit(τ), probability(τ), arrival (τ)} depending on treatment intensity τ
CORR
10. Payoff is *profit times probability* based on treatment minus burn
10. Payoff is *profit times probability* based on treatment minus burn
Now think of it as a game where many participants pick out of the same urn.
Just to offer some ballpark dimensions:
- Arrival time typically measured in years, one or two decades are normal
- Payoffs vary significantly
- Probabilities vary by ε, and typically start in the single digits
- Effect of treatments = ???
- Effect of testing = ???
- Arrival time typically measured in years, one or two decades are normal
- Payoffs vary significantly
- Probabilities vary by ε, and typically start in the single digits
- Effect of treatments = ???
- Effect of testing = ???
Now think of it as a game where multiple participants draw from the same urn.
If you're still with me at this point (which from the responses is pretty much nobody): Congrats! Welcome to my world!
Beers on me... 🍻
Beers on me... 🍻
This is a Silicon Valley venture funnel.
This is a drug discovery funnel.
This is pretty much any corporate R&D pipeline.
It's also an academic paper selection process.
This is a drug discovery funnel.
This is pretty much any corporate R&D pipeline.
It's also an academic paper selection process.
Presidential election races are stochastic funnels.
Baseball farm systems are stochastic funnels.
