Thread by @Meaningness

🧵 Case study in meta-rationality: @slatestarcodex’s fascinating reflections on running a micro-grants program.

Naturally, he wanted to be rational, and found you can’t do this that way, so he got meta-rational (as one should when rationality fails).

astralcodexten.substack.com/p/so-you-want-…

@slatestarcodex “Rationally” would be calculating the probabilistically expected utility of each grant project, and funding proposals accordingly.

You can’t do that because you can’t meaningfully estimate probabilities…

@slatestarcodex … and utility isn’t actually a thing, so you can’t compare dissimilar sorts of projects …

@slatestarcodex … and there are unenumerably many potentially relevant factors, many of them unknown unknowns…

@slatestarcodex So Scott was forced into meta-rationality.

That typically involves socially distributed cognition:

@slatestarcodex Meta-rationality doesn’t mean negating rationality, it means (in part) figuring out how to use it appropriately. Like he did here!

@slatestarcodex Meta-rationality is non-formal ways of thinking about a rational system as a whole. Scott’s post includes a list of ten, many of them very interesting both as examples of meta-rationality and as ways of thinking about non-routine grant-making (something we need much more of!)

@slatestarcodex Figuring out what *not* to reason about is an important aspect of meta-rationality (which, unlike rationality, has to deal somehow or other with unenumerable factors):

@slatestarcodex “When you’re truly lost in a giant multidimensional space, [you have to use] prejudices and heuristics.”

That is: meta-rationality. It is uncomfortable, but necessary, and we can all learn to do it better than we do now.