,
11 tweets,
10 min read
Read on Twitter
@mikedecr 1/n Yeah this is my jam. Thanks for the tag @jim_savage_ . Here's some tips for learning BNP. 1) imo FNBI by Ghosal (linked by @lauretig below) is only good once you have implementation-level knowledge of BNP already and want to fill in theoretical gaps in knowledge.
@mikedecr @jim_savage_ @lauretig 2/n. 2) Muller et al 2015 is great, but short and dense - often acting as a survey with great citations you can dig into. This is a good book to have as reference on your journey. 3) Gelman et al's BDA devotes the last few chapters to some BNP material. Also good for beginners.
@mikedecr @jim_savage_ @lauretig 3/n. 4) BNP is a very large class of models. Packages for implementation tend to be scattered, not well documented, and tedious to use. It's easier to learn if you pick our specific BNP models that interest you.
@mikedecr @jim_savage_ @lauretig 4/n Main ones are: Dirichlet Process (DP) models, tree-based models (e.g. BART), and Gaussian Process models. If interested in survival outcomes, Gamma Process models are essential.
@mikedecr @jim_savage_ @lauretig 5/n For DP models, you can check out my interactive tutorial as you read along Muller: stablemarkets.shinyapps.io/dpmixapp/. Code you can play with is in the GitHub repo. This paper is also good: bit.ly/2ErgiNK - about Chinese Restaurant Processes - closely related to DPs.
@mikedecr @jim_savage_ @lauretig 6/n. For Bayesian Additive Regression Trees (BART), you can check out this paper by Tan and @jasonroy: arxiv.org/abs/1901.07504 . It's very good even at a beginner/intro level level. You can check out original paper here: projecteuclid.org/euclid.aoas/12… .
@mikedecr @jim_savage_ @lauretig 7/n. For Gaussian processes and Gamma Processes, the discussion in BDA for the former and in Muller for the latter is best starting point.
@mikedecr @jim_savage_ @lauretig 8/n. Lastly, BNP is hard to learn because the material is so scattered across conference presentations, short courses, and technical books. Some of these materials have been compiled on Peter Orbanz's site: bit.ly/2BKOos5 . Includes videos, lecture notes, etc.
@mikedecr @jim_savage_ @lauretig 9/n Few words on why BNP is so hard. BNP models involve priors on objects that live in infinite-dimensional spaces. E.g priors on probability measures (the DP). Priors on functions (the Gaussian Process). Priors on trees (BART).
@mikedecr @jim_savage_ @lauretig 10/n. These distributions are well-defined, but they have no density functions with respect to the underlying probability measures. We can write down p(x) where x is real-valued random variable easily, but what is p(T), where T is a random tree?
@mikedecr @jim_savage_ @lauretig n/n. If we can't write down a density, p(T), how do we apply Bayes rule? We often can't in BNP land. Hence we resort to stochastic processes (which use the underlying measures instead of density) to see how realizations from priors translate to realizations from posterior.
