Thread by @ChelseaParlett

Another #ChelseaExplains 🧵 (trying to start with simpler topics).

Today that's 💫Conjugate Priors💫

First, PRIORS. In Bayesian Statistics, we use probability distributions (like a normal, Cauchy, beta...) to represent uncertainty about the value of parameters.

Instead of choosing ONE number for a param, a distribution describes how likely a range of values are

Bayesian Stats works by taking previously known, PRIOR information (this can be from prior data, domain expertise, regularization...) about the parameter

and combining it with data to make the POSTERIOR (the distribution of parameter values AFTER including data)

But that's all conceptual. Math-wise the way this works is by taking the function (the Probability Density Function for continuous, or Probability Mass Function for discrete) for the prior and multiplying it with the function for the likelihood of the data*

* we're ignoring the marginal here (again this tweet is more for statisticians than understanding)🚨

Sometimes multiplying the likelihood and the prior gets...messy (and luckily IRL we have MCMC algos to help us!)

But when doing things by hand, we like cases where the math is easy.

E.g. it would be REALLY NICE if the prior and posterior had the same distribution 💡

Conjugate priors refer to a prior distribution you can use with a specific likelihood function to make the posterior distribution have the SAME type of distribution as the prior!

For example,

a 💫beta prior💫 + a binomial likelihood = 💫beta💫 posterior
a ✨normal✨ prior + a normal likelihood = ✨normal✨ posterior

Now that a lot of Bayesian Stats are done via sampling algorithms like a Gibbs Sampler (JAGS, BUGS) or Hamiltonian Monte Carlo (Stan, PyMC3), we don't have to worry about conjugate priors as much, because we can estimate the posterior no matter if the prior is a conjugate prior

But conjugate priors still provide a nice way to demonstrate concepts, and when appropriate are a LOT QUICKER than a solution found by an MCMC algo, because they have a "closed-form solution" (meaning we can solve for an exact answer)

FIN

again, critiques in GIF form only

( I HATE POSTING formulas and math on Twitter because inevitably I will have made a typo...🥲 But I did it for you #statsTwitter, I did it for you)